[LTER-im-rep] Poll Question concerning LTER Metacat Content and whether it should be archived and hidden during DataONE search and discovery

[Inigo San Gil]

Thanks for your comments, Margaret.

  Am I clear that you think I lack understanding of past and current 
network systems?

  Nice.

Inigo


On 11/12/2015 10:44 AM, Margaret O'Brien wrote:
> Hi Inigo -
> Some of your comments indicate a basic lack of understanding of past 
> and current network systems. I'd like to correct a few of your points 
> here. Please see comments inline.
>
> Thanks -
> Margaret
>
> im-rep at lists.lternet.edu
>
> On 11/12/15 8:50 AM, Inigo San Gil wrote:
>>
>> Linda,
>>
>> Deleted (metacat) metadata should have not been exposed to dataOne in 
>> the first place. I am surprised, even if they call those data 
>> 'archived'.  There are old versions, and then there are deprecated, 
>> deleted data. Any serious IMS can make that distinction. It is good 
>> to see metacat is gone, for that, and many other reasons. (it is not 
>> really gone, though.)
> Metacat was perfectly capable of distinguishing between revisions. FCE 
> had a ad hoc definition of "data set revision" which was inconsistent 
> with what the rest of the network was doing. Unfortunately, their 
> practice didn't get aligned with the rest of the network till after 
> the first DataONE submission. That's why FCE datasets appear the way 
> they do.
>>
>> Also, data that is erased at origin (@ FCE, due to whatever reason), 
>> should be deleted at the public repositories -not sure how-.
> I disagree. data that are in repositories should have undergone 
> sufficient review to be appropriate for the public before it was 
> submitted. There exists a manual process for removal (for rare cases).
>>
>> Due to shortcomings with the repository systems we have been using, 
>> this deletion has been a headache. Always.
> Removing datasets is not a common occurrence, so should not be a 
> trivial task.
>> At best, we were forced by default to retire our numerical 
>> identifiers, this has been quite irritating.  At worse, unwanted, bad 
>> data remains public through these repos.
> Identifiers, if they were carefully assigned originally and in a 
> robust system, were not retired. See any SBC dataset in PASTA, and you 
> will see that the same identifier was used for previous revisions in 
> Metacat. I know this is true for at least half the sites as well (GCE, 
> MCR, HFR, VCR, AND, NTL among others).
>>
>>
>> There is a silver lining to all this, old versions are just that, old 
>> versions.  Anybody that is able to get the hands on an old version of 
>> data ought to realize that this is not the most current version.
> This is true, and has always been how our systems work (metacat and 
> pasta).
>>
>>
>> I would ask dataOne to erase that content if you are concerned.
> It will be archived (not erased), so that if someone has a link, it 
> will still work
>>
>> Cheers, Inigo
>>
>>
>>
>> On 11/12/2015 8:58 AM, Linda A Powell wrote:
>>>
>>> Dear Information Managers,
>>>
>>> As some of you may know, the FCE LTER program discontinued its 
>>> practice of file versioning where each updated data file would be 
>>> given a different file name (.v1, .v2, etc.) and a new EML package 
>>> ID.     We initially had 525 data files that got combined into new 
>>> data files so our FCE data count decreased to 125 data files.  I 
>>> started the EML packaging ID numbers for the newly combined data 
>>> files at knb-lter-fce.1050 (well beyond the last package ID 
>>> knb-lter-fce.525) and I personally deleted all the old versioned 
>>> data from the LTER Metacat.  I then added the 125 new files back 
>>> into the Metacat and PASTA.
>>>
>>>
>>> Unfortunately,  those files were never really deleted from Metacat, 
>>> only archived, and when the Metacat files were harvested into 
>>> DataONE, *ALL* my files, including those I thought were ‘deleted’ 
>>> and the existing, were uploaded.  Now the FCE has a big mess!  The 
>>> old 'deleted' files are listed but none of the files exist any 
>>> longer so the links to the data don’t work.  I’m sure the DataONE 
>>> users are frustrated! There may be Metacat files that other IMs have 
>>> thought were deleted that are also showing up in DataONE.
>>>
>>>
>>> */My question to the LTER IMs is whether the content that existed in 
>>> the LTER Metacat should be archived and made hidden during search 
>>> and discovery from the DataONE infrastructure (i.e. ONEMercury, CN 
>>> API, etc.)? /* I've asked Mark Servilla to help with this issue and 
>>> we thought It would be simplest and scale economically if he could 
>>> perform this operation at one time and for all LTER site content as 
>>> opposed to performing this operation for each site independently.  
>>> Of course we want input from the IMC before we move forward.
>>>
>>>
>>> *I’ve created a Doodle Poll 
>>> (http://doodle.com/poll/uw87khqqyhhsiry5) and would appreciate input 
>>> from EACH of the LTER site IMs as to whether the content that 
>>> existed in the LTER Metacat should be archived and made hidden 
>>> during search and discovery from the DataONE infrastructure?  Please 
>>> select ‘Yes’ or ‘No’. *
>>>
>>> Thank you in advance for your participation!
>>>
>>> Kindest Regards,
>>>
>>> Linda
>>>
>>>
>>> Linda Powell
>>> Information Manager
>>> Florida Coastal Everglades LTER Program
>>> OE 148, Florida International University
>>> University Park
>>> Miami, Florida 33199
>>> Phone (Tallahassee, FL): 850-745-0381
>>> Phone(Miami,FL): 305-856-0039 or  305-348-6054
>>> Website: http://fcelter.fiu.edu
>>>
>>>
>>> _______________________________________________
>>> Long Term Ecological Research Network
>>> im-rep mailing list
>>> im-rep at lternet.edu
>>
>>
>>
>
> -----------
> Margaret O'Brien
> Information Management
> Santa Barbara Coastal LTER
> Marine Science Institute, UCSB
> Santa Barbara, CA 93106
> 805-893-2071 (voice)
> http://sbc.lternet.edu
>
>> _______________________________________________
>> Long Term Ecological Research Network
>> im-rep mailing list
>> im-rep at lternet.edu
>
> _______________________________________________
> Long Term Ecological Research Network
> im-rep mailing list
> im-rep at lternet.edu