[LTER-im-rep] Poll Question concerning LTER Metacat Content andwhether it should be archived and hidden during DataONE search anddiscovery

[Ken Ramsey]

Hi Margaret,

I would appreciate a more open approach to communication than chiding
us when you think we are wrong or mistaken. Given most of us were not
involved in the decisions and planning you refer, it would be great if
you listen to us this time. We used to be able to communicate without
criticism.

We cannot all be in tight control of our IDs like you and it extremely
embarrassing to have outdated data and metadata available in DataOne
that has since been corrected in PASTA. All of the Jornada datasets from
Metacat are out of date and contain data and metadata problems that have
been fixed in the version now coming from PASTA. None of our old data
should be publicly available any longer. I stressed this point a few
years ago with Mark and DataOne with no luck.

Ken

>>> Inigo San Gil <isangil at lternet.edu> 2015-11-12 11:12 AM >>>

Thanks for your comments, Margaret.

  Am I clear that you think I lack understanding of past and current 
network systems?

  Nice.

Inigo


On 11/12/2015 10:44 AM, Margaret O'Brien wrote:
> Hi Inigo -
> Some of your comments indicate a basic lack of understanding of past

> and current network systems. I'd like to correct a few of your points

> here. Please see comments inline.
>
> Thanks -
> Margaret
>
> im-rep at lists.lternet.edu 
>
> On 11/12/15 8:50 AM, Inigo San Gil wrote:
>>
>> Linda,
>>
>> Deleted (metacat) metadata should have not been exposed to dataOne
in 
>> the first place. I am surprised, even if they call those data 
>> 'archived'.  There are old versions, and then there are deprecated,

>> deleted data. Any serious IMS can make that distinction. It is good

>> to see metacat is gone, for that, and many other reasons. (it is not

>> really gone, though.)
> Metacat was perfectly capable of distinguishing between revisions.
FCE 
> had a ad hoc definition of "data set revision" which was inconsistent

> with what the rest of the network was doing. Unfortunately, their 
> practice didn't get aligned with the rest of the network till after 
> the first DataONE submission. That's why FCE datasets appear the way

> they do.
>>
>> Also, data that is erased at origin (@ FCE, due to whatever reason),

>> should be deleted at the public repositories -not sure how-.
> I disagree. data that are in repositories should have undergone 
> sufficient review to be appropriate for the public before it was 
> submitted. There exists a manual process for removal (for rare
cases).
>>
>> Due to shortcomings with the repository systems we have been using,

>> this deletion has been a headache. Always.
> Removing datasets is not a common occurrence, so should not be a 
> trivial task.
>> At best, we were forced by default to retire our numerical 
>> identifiers, this has been quite irritating.  At worse, unwanted,
bad 
>> data remains public through these repos.
> Identifiers, if they were carefully assigned originally and in a 
> robust system, were not retired. See any SBC dataset in PASTA, and
you 
> will see that the same identifier was used for previous revisions in

> Metacat. I know this is true for at least half the sites as well
(GCE, 
> MCR, HFR, VCR, AND, NTL among others).
>>
>>
>> There is a silver lining to all this, old versions are just that,
old 
>> versions.  Anybody that is able to get the hands on an old version
of 
>> data ought to realize that this is not the most current version.
> This is true, and has always been how our systems work (metacat and 
> pasta).
>>
>>
>> I would ask dataOne to erase that content if you are concerned.
> It will be archived (not erased), so that if someone has a link, it 
> will still work
>>
>> Cheers, Inigo
>>
>>
>>
>> On 11/12/2015 8:58 AM, Linda A Powell wrote:
>>>
>>> Dear Information Managers,
>>>
>>> As some of you may know, the FCE LTER program discontinued its 
>>> practice of file versioning where each updated data file would be 
>>> given a different file name (.v1, .v2, etc.) and a new EML package

>>> ID.     We initially had 525 data files that got combined into new

>>> data files so our FCE data count decreased to 125 data files.  I 
>>> started the EML packaging ID numbers for the newly combined data 
>>> files at knb-lter-fce.1050 (well beyond the last package ID 
>>> knb-lter-fce.525) and I personally deleted all the old versioned 
>>> data from the LTER Metacat.  I then added the 125 new files back 
>>> into the Metacat and PASTA.
>>>
>>>
>>> Unfortunately,  those files were never really deleted from Metacat,

>>> only archived, and when the Metacat files were harvested into 
>>> DataONE, *ALL* my files, including those I thought were
‘deleted’ 
>>> and the existing, were uploaded.  Now the FCE has a big mess!  The

>>> old 'deleted' files are listed but none of the files exist any 
>>> longer so the links to the data don’t work.  I’m sure the
DataONE 
>>> users are frustrated! There may be Metacat files that other IMs
have 
>>> thought were deleted that are also showing up in DataONE.
>>>
>>>
>>> */My question to the LTER IMs is whether the content that existed
in 
>>> the LTER Metacat should be archived and made hidden during search 
>>> and discovery from the DataONE infrastructure (i.e. ONEMercury, CN

>>> API, etc.)? /* I've asked Mark Servilla to help with this issue and

>>> we thought It would be simplest and scale economically if he could

>>> perform this operation at one time and for all LTER site content as

>>> opposed to performing this operation for each site independently. 

>>> Of course we want input from the IMC before we move forward.
>>>
>>>
>>> *I’ve created a Doodle Poll 
>>> (http://doodle.com/poll/uw87khqqyhhsiry5) and would appreciate
input 
>>> from EACH of the LTER site IMs as to whether the content that 
>>> existed in the LTER Metacat should be archived and made hidden 
>>> during search and discovery from the DataONE infrastructure? 
Please 
>>> select ‘Yes’ or ‘No’. *
>>>
>>> Thank you in advance for your participation!
>>>
>>> Kindest Regards,
>>>
>>> Linda
>>>
>>>
>>> Linda Powell
>>> Information Manager
>>> Florida Coastal Everglades LTER Program
>>> OE 148, Florida International University
>>> University Park
>>> Miami, Florida 33199
>>> Phone (Tallahassee, FL): 850-745-0381
>>> Phone(Miami,FL): 305-856-0039 or  305-348-6054
>>> Website: http://fcelter.fiu.edu 
>>>
>>>
>>> _______________________________________________
>>> Long Term Ecological Research Network
>>> im-rep mailing list
>>> im-rep at lternet.edu 
>>
>>
>>
>
> -----------
> Margaret O'Brien
> Information Management
> Santa Barbara Coastal LTER
> Marine Science Institute, UCSB
> Santa Barbara, CA 93106
> 805-893-2071 (voice)
> http://sbc.lternet.edu 
>
>> _______________________________________________
>> Long Term Ecological Research Network
>> im-rep mailing list
>> im-rep at lternet.edu 
>
> _______________________________________________
> Long Term Ecological Research Network
> im-rep mailing list
> im-rep at lternet.edu 

_______________________________________________
Long Term Ecological Research Network
im-rep mailing list
im-rep at lternet.edu