[LTER-im-rep] Poll Question concerning LTER Metacat Content and whether it should be archived and hidden during DataONE search and discovery

[Margaret O'Brien]

Hi Inigo -
Some of your comments indicate a basic lack of understanding of past and 
current network systems. I'd like to correct a few of your points here. 
Please see comments inline.

Thanks -
Margaret

im-rep at lists.lternet.edu

On 11/12/15 8:50 AM, Inigo San Gil wrote:
>
> Linda,
>
> Deleted (metacat) metadata should have not been exposed to dataOne in 
> the first place. I am surprised, even if they call those data 
> 'archived'.  There are old versions, and then there are deprecated, 
> deleted data. Any serious IMS can make that distinction. It is good to 
> see metacat is gone, for that, and many other reasons. (it is not 
> really gone, though.)
Metacat was perfectly capable of distinguishing between revisions. FCE 
had a ad hoc definition of "data set revision" which was inconsistent 
with what the rest of the network was doing. Unfortunately, their 
practice didn't get aligned with the rest of the network till after the 
first DataONE submission. That's why FCE datasets appear the way they do.
>
> Also, data that is erased at origin (@ FCE, due to whatever reason), 
> should be deleted at the public repositories -not sure how-.
I disagree. data that are in repositories should have undergone 
sufficient review to be appropriate for the public before it was 
submitted. There exists a manual process for removal (for rare cases).
>
> Due to shortcomings with the repository systems we have been using, 
> this deletion has been a headache. Always.
Removing datasets is not a common occurrence, so should not be a trivial 
task.
> At best, we were forced by default to retire our numerical 
> identifiers, this has been quite irritating.  At worse, unwanted, bad 
> data remains public through these repos.
Identifiers, if they were carefully assigned originally and in a robust 
system, were not retired. See any SBC dataset in PASTA, and you will see 
that the same identifier was used for previous revisions in Metacat. I 
know this is true for at least half the sites as well (GCE, MCR, HFR, 
VCR, AND, NTL among others).
>
>
> There is a silver lining to all this, old versions are just that, old 
> versions.  Anybody that is able to get the hands on an old version of 
> data ought to realize that this is not the most current version.
This is true, and has always been how our systems work (metacat and pasta).
>
>
> I would ask dataOne to erase that content if you are concerned.
It will be archived (not erased), so that if someone has a link, it will 
still work
>
> Cheers, Inigo
>
>
>
> On 11/12/2015 8:58 AM, Linda A Powell wrote:
>>
>> Dear Information Managers,
>>
>> As some of you may know, the FCE LTER program discontinued its 
>> practice of file versioning where each updated data file would be 
>> given a different file name (.v1, .v2, etc.) and a new EML package 
>> ID.     We initially had 525 data files that got combined into new 
>> data files so our FCE data count decreased to 125 data files.  I 
>> started the EML packaging ID numbers for the newly combined data 
>> files at knb-lter-fce.1050 (well beyond the last package ID 
>> knb-lter-fce.525) and I personally deleted all the old versioned data 
>> from the LTER Metacat.  I then added the 125 new files back into the 
>> Metacat and PASTA.
>>
>>
>> Unfortunately,  those files were never really deleted from Metacat, 
>> only archived, and when the Metacat files were harvested into 
>> DataONE, *ALL* my files, including those I thought were ‘deleted’ and 
>> the existing, were uploaded.  Now the FCE has a big mess!  The old 
>> 'deleted' files are listed but none of the files exist any longer so 
>> the links to the data don’t work.  I’m sure the DataONE users are 
>> frustrated!  There may be Metacat files that other IMs have thought 
>> were deleted that are also showing up in DataONE.
>>
>>
>> */My question to the LTER IMs is whether the content that existed in 
>> the LTER Metacat should be archived and made hidden during search and 
>> discovery from the DataONE infrastructure (i.e. ONEMercury, CN API, 
>> etc.)? /* I've asked Mark Servilla to help with this issue and we 
>> thought It would be simplest and scale economically if he could 
>> perform this operation at one time and for all LTER site content as 
>> opposed to performing this operation for each site independently.  Of 
>> course we want input from the IMC before we move forward.
>>
>>
>> *I’ve created a Doodle Poll (http://doodle.com/poll/uw87khqqyhhsiry5) 
>> and would appreciate input from EACH of the LTER site IMs as to 
>> whether the content that existed in the LTER Metacat should be 
>> archived and made hidden during search and discovery from the DataONE 
>> infrastructure?  Please select ‘Yes’ or ‘No’. *
>>
>> Thank you in advance for your participation!
>>
>> Kindest Regards,
>>
>> Linda
>>
>>
>> Linda Powell
>> Information Manager
>> Florida Coastal Everglades LTER Program
>> OE 148, Florida International University
>> University Park
>> Miami, Florida 33199
>> Phone (Tallahassee, FL): 850-745-0381
>> Phone(Miami,FL): 305-856-0039 or  305-348-6054
>> Website: http://fcelter.fiu.edu
>>
>>
>> _______________________________________________
>> Long Term Ecological Research Network
>> im-rep mailing list
>> im-rep at lternet.edu
>
>
>

-----------
Margaret O'Brien
Information Management
Santa Barbara Coastal LTER
Marine Science Institute, UCSB
Santa Barbara, CA 93106
805-893-2071 (voice)
http://sbc.lternet.edu

> _______________________________________________
> Long Term Ecological Research Network
> im-rep mailing list
> im-rep at lternet.edu