<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
span.hoenzb
{mso-style-name:hoenzb;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;}
span.m8565574059455562412hoenzb
{mso-style-name:m_8565574059455562412hoenzb;}
span.EmailStyle22
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi Jonathan et al.,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Once you get to think about this, would you be interested in summarizing this as best practice documentation? It sounds as so everyone has slightly, but not
majorly different approaches that would be great to compile in an article. I am sure Don and John would help – right?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I think this discussion goes beyond the simple provenance implementation in EML where we say in our best practices:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Arial","sans-serif"">The <dataSource> tag is for nesting an EML dataset which may be an input to a <methodStep> of the data being described, e.g., calibration information for an instrument or input
parameters for a model. This element will also be used by the PASTA provenance tracking system for recording the source data when a derived product is created and described with EML. For more information, see Section III, Recommendations for
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Arial","sans-serif"">Compatibility with External Applications.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">And an example of the EML is on page 51 of the document
<a href="https://im.lternet.edu/sites/im.lternet.edu/files/emlbestpractices-2.0-FINAL-20110801_0.pdf">
https://im.lternet.edu/sites/im.lternet.edu/files/emlbestpractices-2.0-FINAL-20110801_0.pdf</a>
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">and somebody mentioned already that if the source dataset is in PASTA it will generate such an EML snippet. Otherwise you’d have to write it yourself.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Corinna<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> im [mailto:im-bounces@lists.lternet.edu]
<b>On Behalf Of </b>Jonathan Walsh<br>
<b>Sent:</b> Thursday, September 21, 2017 10:49 AM<br>
<b>To:</b> Porter, John Henderson (jhp7e)<br>
<b>Cc:</b> Emma Rosi; IM committee; jhp7e<br>
<b>Subject:</b> Re: [LTER-im] Representing datasets from other parties when they are integral to your work<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Thanks for all your thoughtful replies. You've given me plenty to read, plenty of examples, and plenty of ideas to think about. I'm going through the ideas and suggestions as time permits in between proposal-generating work!<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Thu, Sep 21, 2017 at 11:30 AM, Jonathan Walsh <<a href="mailto:walshjcaryinstitute@gmail.com" target="_blank">walshjcaryinstitute@gmail.com</a>> wrote:<o:p></o:p></p>
<div>
<p class="MsoNormal">John,<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I like the idea of using a methodstep and no matter what else I do or don't' do I'm definitely doing that.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#888888"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#888888">Jonathan<o:p></o:p></span></p>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Wed, Sep 20, 2017 at 3:37 PM, Porter, John Henderson (jhp7e) <<a href="mailto:jhp7e@eservices.virginia.edu" target="_blank">jhp7e@eservices.virginia.edu</a>> wrote:<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal">Interesting distinction..... <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">We have our local catalog, the LTER Data Portal/EDI, to which we send our metadata (and PASTA then fetches the data). DataONE gets the metadata from PASTA, then links back to PASTA for the data itself. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">It looks as if NSF will be increasingly emphasizing having data on EDI/PASTA, and deemphasizing the local catalog. And as we showed at ESIP, you can also implement a local view of the metadata/data in PASTA as a local catalog.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Back to the original subject, we generally try to avoid serving data that is also served by others because we want to avoid the headaches of keeping it updated, and the potential confusion of users over who to cite.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">-JP<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Sep 20, 2017 1:46 PM, Jonathan Walsh <<a href="mailto:walshjcaryinstitute@gmail.com" target="_blank">walshjcaryinstitute@gmail.com</a>> wrote:<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal">Thank you for replying. One clarification: <o:p></o:p></p>
<div>
<p class="MsoNormal">>>>><span style="font-size:9.5pt">For LTER data, of course, we are the "best source" so we document that data and share it via PASTA, DataONE etc. </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.5pt">Does that mean you submit a copy of that data to PASTA, DataONE, etc, or just some documentation of what data it is and how to find it?</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.5pt">Thanks!</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Wed, Sep 20, 2017 at 1:12 PM, John Porter <<a href="mailto:jhp7e@eservices.virginia.edu" target="_blank">jhp7e@eservices.virginia.edu</a>> wrote:<o:p></o:p></p>
<div>
<p>Jonathan,<o:p></o:p></p>
<p>That sounds similar to our approach. <o:p></o:p></p>
<p>Generally speaking there is a "best source" for data - the place where updated data is reliably available - and in the case of USGS that is their web site, so we just point people there. For LTER data, of course, we are the "best source" so we document
that data and share it via PASTA, DataONE etc. <o:p></o:p></p>
<p>There are a few cases where we do maintain "local" copies of external data, either because the availability at the source is unreliable (not the case for USGS), the data volume is best dealt with on a local network (e.g., large LiDAR datasets), the data
is really hard to locate and extract, or where there is a reason that you want to maintain a particular static version of the data (e.g., for a series of analyses conducted over a period of time where you don't want the underlying data to change). However,
we DON'T include them in the data catalog. <o:p></o:p></p>
<p>Don't forget that for your derived products you can include references to the USGS source in a "methodStep" in EML. For referring to source data already documented in EML and stored in PASTA via the "provenance" web service (e.g.,
<a href="https://pasta.lternet.edu/package/provenance/eml/knb-lter-bes/332/580" target="_blank">
https://pasta.lternet.edu/package/provenance/eml/knb-lter-bes/332/580</a>) which will produce a methodStep "stub" that can easily be included in your metadata. For USGS you'll need to generate your own methodStep - but you can use the example produced by the
provenance web service to do that....<o:p></o:p></p>
<p>Hope that helps!<o:p></o:p></p>
<p> -John Porter<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On 9/20/2017 11:11 AM, Jonathan Walsh wrote:<o:p></o:p></p>
</div>
</div>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal">Hi IMs! <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I have a question on how to best represent data that your study uses, but is provided by others. If you use such data, we could use your insights on how you make that portion of your data available to the community and the LTER.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Baltimore Ecosystem Study gets its stream flow data from the USGS. We in turn use this flow data to calculate our daily loads and other results that we track. The USGS data are kept on their website and we incorporate them into our work.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">For the purposes of making our data available to the larger community, (LTER, PASTA, DataONE, etc.) we have historically just pointed to the USGS data on the USGS site. (example: <a href="https://waterdata.usgs.gov/usa/nwis/uv?01589197" target="_blank">https://waterdata.usgs.gov/usa/nwis/uv?01589197</a> )
as opposed to collecting our own copy and providing it ourselves.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The above precludes us from providing a direct link to the data such as that that would be PASTA "type 1".<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">If you have any suggestions as to how, if differently, we should represent these data, which are integral to our work, but not provided by us, I would very much appreciate hearing them.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thank you!<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Jonathan<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">- <o:p></o:p></p>
</div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal">Jonathan Walsh<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><a href="http://orcid.org/0000-0002-0658-0814" target="_blank"><span style="font-size:11.5pt;font-family:"Helvetica","sans-serif";color:#1155CC;background:#FFEE94">orcid</span><span style="font-size:11.5pt;font-family:"Helvetica","sans-serif";color:#1155CC">.org/0000-0002-0658-0814</span></a><br>
Information Manager, Baltimore Ecosystem Study<br>
Cary Institute of Ecosystem Studies<br>
Box AB; Route 44A<br>
Millbrook, NY 12545-0129<br>
P: <a href="tel:845%2F677%2F7600%20Extension%20103" target="_blank">845/677/7600 Extension 103</a><br>
F: <a href="tel:845%2F677%2F5976" target="_blank">845/677/5976</a><br>
E: <a href="mailto:WalshJ@caryinstitute.org" target="_blank">WalshJ@caryinstitute.org</a><o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
</div>
</div>
<pre>_______________________________________________<o:p></o:p></pre>
<pre>Long Term Ecological Research Network<o:p></o:p></pre>
<pre>im mailing list<o:p></o:p></pre>
<pre><a href="mailto:im@lternet.edu" target="_blank">im@lternet.edu</a><o:p></o:p></pre>
<pre><o:p> </o:p></pre>
</blockquote>
<p class="MsoNormal"><span class="m8565574059455562412hoenzb"><span style="color:#888888"><o:p> </o:p></span></span></p>
<pre><span style="color:#888888">-- <o:p></o:p></span></pre>
<pre><span style="color:#888888">John H. Porter<o:p></o:p></span></pre>
<pre><span style="color:#888888">Dept. of Environmental Sciences<o:p></o:p></span></pre>
<pre><span style="color:#888888">University of Virginia<o:p></o:p></span></pre>
<pre><span style="color:#888888"><a href="https://maps.google.com/?q=291+McCormick+Road&entry=gmail&source=g" target="_blank">291 McCormick Road</a><o:p></o:p></span></pre>
<pre><span style="color:#888888">PO Box 400123<o:p></o:p></span></pre>
<pre><span style="color:#888888">Charlottesville, VA 22904-4123<o:p></o:p></span></pre>
<pre><span style="color:#888888">ORCID: <a href="http://orcid.org/0000-0003-3118-5784" target="_blank">http://orcid.org/0000-0003-3118-5784</a></span><o:p></o:p></pre>
</div>
</div>
<p class="MsoNormal"><span style="color:#888888"><br>
<br clear="all">
</span><span class="m8565574059455562412hoenzb"><o:p></o:p></span></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><span class="m8565574059455562412hoenzb"><span style="color:#888888">--
</span></span><o:p></o:p></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal">Jonathan Walsh<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><a href="http://orcid.org/0000-0002-0658-0814" target="_blank"><span style="font-size:11.5pt;font-family:"Helvetica","sans-serif";color:#1155CC;background:#FFEE94">orcid</span><span style="font-size:11.5pt;font-family:"Helvetica","sans-serif";color:#1155CC">.org/0000-0002-0658-0814</span></a><br>
Information Manager, Baltimore Ecosystem Study<br>
Cary Institute of Ecosystem Studies<br>
Box AB; Route 44A<br>
Millbrook, NY 12545-0129<br>
P: <a href="tel:845%2F677%2F7600%20Extension%20103" target="_blank">845/677/7600 Extension 103</a><br>
F: <a href="tel:845%2F677%2F5976" target="_blank">845/677/5976</a><br>
E: <a href="mailto:WalshJ@caryinstitute.org" target="_blank">WalshJ@caryinstitute.org</a><o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
<p class="MsoNormal"><br>
<br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <o:p></o:p></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal">Jonathan Walsh<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><a href="http://orcid.org/0000-0002-0658-0814" target="_blank"><span style="font-size:11.5pt;font-family:"Helvetica","sans-serif";color:#1155CC;background:#FFEE94">orcid</span><span style="font-size:11.5pt;font-family:"Helvetica","sans-serif";color:#1155CC">.org/0000-0002-0658-0814</span></a><br>
Information Manager, Baltimore Ecosystem Study<br>
Cary Institute of Ecosystem Studies<br>
Box AB; Route 44A<br>
Millbrook, NY 12545-0129<br>
P: <a href="tel:845%2F677%2F7600%20Extension%20103" target="_blank">845/677/7600 Extension 103</a><br>
F: <a href="tel:845%2F677%2F5976" target="_blank">845/677/5976</a><br>
E: <a href="mailto:WalshJ@caryinstitute.org" target="_blank">WalshJ@caryinstitute.org</a><o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><br>
<br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <o:p></o:p></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal">Jonathan Walsh<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><a href="http://orcid.org/0000-0002-0658-0814" target="_blank"><span style="font-size:11.5pt;font-family:"Helvetica","sans-serif";color:#1155CC;background:#FFEE94">orcid</span><span style="font-size:11.5pt;font-family:"Helvetica","sans-serif";color:#1155CC">.org/0000-0002-0658-0814</span></a><br>
Information Manager, Baltimore Ecosystem Study<br>
Cary Institute of Ecosystem Studies<br>
Box AB; Route 44A<br>
Millbrook, NY 12545-0129<br>
P: <a href="tel:845%2F677%2F7600%20Extension%20103" target="_blank">845/677/7600 Extension 103</a><br>
F: <a href="tel:845%2F677%2F5976" target="_blank">845/677/5976</a><br>
E: <a href="mailto:WalshJ@caryinstitute.org" target="_blank">WalshJ@caryinstitute.org</a><o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>