<div dir="ltr">Don,<div><br></div><div>Your approach makes a lot of sense to me. I appreciate the information. In the case of BES this would be like you say, a good use of the method step node in EML.</div><div><br></div><div>Thanks! </div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Sep 20, 2017 at 3:52 PM, Henshaw, Donald <span dir="ltr"><<a href="mailto:don.henshaw@oregonstate.edu" target="_blank">don.henshaw@oregonstate.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div class="m_-8918947396652542444WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hi,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">The USGS collects our streamflow for Lookout Creek which is the highest order stream in the Andrews. We do repackage the USGS data and include this in the same
data set with the same structure as all of the other nine small watersheds where we collect streamflow. We do this for several reasons:<u></u><u></u></span></p>
<p class="m_-8918947396652542444MsoListParagraph"><u></u><span style="font-size:11.0pt;font-family:Symbol;color:#1f497d"><span>·<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">As a convenience for our PIs the AND LTER provides both the high temporal and daily versions of the USGS streamflow data in the same format as all
of our other watersheds.<u></u><u></u></span></p>
<p class="m_-8918947396652542444MsoListParagraph"><u></u><span style="font-size:11.0pt;font-family:Symbol;color:#1f497d"><span>·<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Maintaining the USGS streamflow in AND LTER formats allows our applications for summarizing streamflow and preparing streamflow data for use with
our stream chemistry samples work for these USGS data<u></u><u></u></span></p>
<p class="m_-8918947396652542444MsoListParagraph"><u></u><span style="font-size:11.0pt;font-family:Symbol;color:#1f497d"><span>·<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">The high temporal versions are only available for the past 15 years or so through USGS. The AND LTER has recreated an hourly data set after historic
reconstruction from charts, punch tapes and printouts going back to 1949, so we offer the only complete high temporal version of this data. The USGS never bothered to Q/C any of the high temporal resolution data until more recent years.
<u></u><u></u></span></p>
<p class="m_-8918947396652542444MsoListParagraph"><u></u><span style="font-size:11.0pt;font-family:Symbol;color:#1f497d"><span>·<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">The USFS maintained this gage for eight years in the 1950s and 60s and the USGS does not maintain these records. Our daily record is the same as the
USGS daily record beginning in 1949 except we include these additional 8 years, so we offer the most complete and “best source” for the daily record as well<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">We do reference the USGS contribution in the data abstract and methods, but including specific references to the USGS source in a method step in EML sounds
very appropriate.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">On the other hand we have never done anything with our National Atmospheric Deposition Program (NADP) data, although we do collect similar precipitation chemistry
data at AND LTER. This NADP data is well maintained on their website beginning in 1980. However Andrews personnel have spent time every week sending information and samples to NADP since 1980 we should probably make this data more obviously available.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Don<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> im [mailto:<a href="mailto:im-bounces@lists.lternet.edu" target="_blank">im-bounces@lists.<wbr>lternet.edu</a>]
<b>On Behalf Of </b>Margaret O'Brien<br>
<b>Sent:</b> Wednesday, September 20, 2017 11:15 AM<br>
<b>To:</b> Jonathan Walsh<br>
<b>Cc:</b> Emma Rosi; IM committee<br>
<b>Subject:</b> Re: [LTER-im] Representing datasets from other parties when they are integral to your work<u></u><u></u></span></p><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">Hi Jonathan -<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I see several ways to answer this question. Ideally, there is a way to handle different sources of data without complicating your IM system too much. And I hope this answer is not too complicated, either.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">It's pretty simple to link to other sources. SBC LTER does it on the main data page (<a href="http://sbc.lternet.edu/data/" target="_blank">http://sbc.lternet.edu/data/</a>)<wbr>. But some of these we use for our LTER research, so they have
been turned into datasets with EML, too. At SBC LTER, we call this "exogenous data", or "Type 0". And so our re-packaging turns it into "Type I". But we do try to make it clear where it came from. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Particularly in the watersheds, there are a number of these, including from the USGS. I should add though, that we don't get the USGS data that we use from their website, we get their highest resolution data thru back-channels at the end
of the water year after it's been QC'd. In total, SBC LTER has 30-40 datasets like this, including both precipitation and stream flow.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">For stream flow and precip, we use these rules for creating datasets from exogenous dat. Keep in mind that the most visible parts of a citation are the creators and title, so these are important.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">1. the data are reformatted to match our own (so they can be used together - now they are Type 1). Since stream flow is calc'd from stage height, and we do that part here, we do call it "ours"<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">2. creator is the primary PI conducting that part of the project<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">3. the dataset title names the source (if not us), and has both the original station and our station id in it. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">4. the abstract also names the source<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Here are some examples:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">our data from a USGS stream gauge:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><a href="http://sbc.lternet.edu/cgi-bin/showDataset.cgi?docid=knb-lter-sbc.3018" target="_blank">http://sbc.lternet.edu/cgi-<wbr>bin/showDataset.cgi?docid=knb-<wbr>lter-sbc.3018</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">A similar dataset from one of our own height-gauges:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><a href="http://sbc.lternet.edu/cgi-bin/showDataset.cgi?docid=knb-lter-sbc.3007" target="_blank">http://sbc.lternet.edu/cgi-<wbr>bin/showDataset.cgi?docid=knb-<wbr>lter-sbc.3007</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">precipitation data collected by the county:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><a href="http://sbc.lternet.edu/cgi-bin/showDataset.cgi?docid=knb-lter-sbc.5012" target="_blank">http://sbc.lternet.edu/cgi-<wbr>bin/showDataset.cgi?docid=knb-<wbr>lter-sbc.5012</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">We have other exogenous data where we did nothing to it at all -- all we do is repost. These are mostly for reference, and for those, we make no claim to this data (unless we treated it somehow, like aggregating).<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">So those rules:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">1. data are in original format<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">2. creator is original org (not us)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">3. dataset title shows is as close as we can get to what it was when received or downloaded. It may take a phone call to get it right.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">4. contact - add one for the org, too.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">e.g., here is some KML data, that describes the perimeter of a recent fire. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><a href="http://sbc.lternet.edu/cgi-bin/showDataset.cgi?docid=knb-lter-sbc.70" target="_blank">http://sbc.lternet.edu/cgi-<wbr>bin/showDataset.cgi?docid=knb-<wbr>lter-sbc.70</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Another project I work with is a biodiversity observation network (BON). Most of the data they work with are exogenous, and a lot of what the BON does is to curate that data so they can use it in integrated research, and create datasets
along the way. They have adopted the LTER data management protocols, including the whole EML > EDI > DataONE pathway.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">So we highlight the packaging guidelines for different types (de novo, curated exogenous, integration products)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">For that, see "Data Packaging", here: <a href="http://sbc.marinebon.org/data/overview/" target="_blank">
http://sbc.marinebon.org/data/<wbr>overview/</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">These are very similar to what we do with LTER, but the process is more formalized. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">One more note:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">EML has an area for describing "source data", under methods. Use it if you can. There is a node called sourceData, that works for holding info about another PASTA dataset. For URLs that are non-pasta, the only current option is to use the
text fields. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Here is an example for one of the biodiversity datasets that integrates data from 4 projects (open up Metadat > Methods):<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><a href="https://portal.edirepository.org/nis/metadataviewer?packageid=edi.5.2" target="_blank">https://portal.edirepository.<wbr>org/nis/metadataviewer?<wbr>packageid=edi.5.2</a><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Yes, the LTER dataset is in PASTA, but our code was not not sophisticated enough to get the id inserted into edi.5's /eml//dataSource node. But since both the LTER and BON data are time-series, we'll get that in on the next update.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Margaret<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><br clear="all">
<u></u><u></u></p>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">Margaret O'Brien<br>
ORCID: 0000-0002-1693-8322<br>
Information Management<br>
Marine Science Institute, UCSB<br>
Santa Barbara, CA 93106<br>
<a href="tel:(805)%20893-2071" target="_blank">805-893-2071</a> (voice)<br>
<a href="http://environmentaldatainitiative.org" target="_blank">http://<wbr>environmentaldatainitiative.<wbr>org</a><br>
<a href="http://sbc.marinebon.org" target="_blank">http://sbc.marinebon.org</a></span><u></u><u></u></p>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New""><a href="http://sbc.lternet.edu" target="_blank">http://sbc.lternet.edu</a></span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<pre><u></u> <u></u></pre>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Wed, Sep 20, 2017 at 8:11 AM, Jonathan Walsh <<a href="mailto:walshjcaryinstitute@gmail.com" target="_blank">walshjcaryinstitute@gmail.com</a><wbr>> wrote:<u></u><u></u></p>
<div>
<p class="MsoNormal">Hi IMs!<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I have a question on how to best represent data that your study uses, but is provided by others. If you use such data, we could use your insights on how you make that portion of your data available to the community and the LTER.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Baltimore Ecosystem Study gets its stream flow data from the USGS. We in turn use this flow data to calculate our daily loads and other results that we track. The USGS data are kept on their website and we incorporate them into our work.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">For the purposes of making our data available to the larger community, (LTER, PASTA, DataONE, etc.) we have historically just pointed to the USGS data on the USGS site. (example: <a href="https://waterdata.usgs.gov/usa/nwis/uv?01589197" target="_blank">https://waterdata.usgs.gov/<wbr>usa/nwis/uv?01589197</a> )
as opposed to collecting our own copy and providing it ourselves.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">The above precludes us from providing a direct link to the data such as that that would be PASTA "type 1".<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">If you have any suggestions as to how, if differently, we should represent these data, which are integral to our work, but not provided by us, I would very much appreciate hearing them.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Thank you!<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Jonathan<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">- <u></u><u></u></p>
</div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal">Jonathan Walsh<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><a href="http://orcid.org/0000-0002-0658-0814" target="_blank"><span style="font-size:11.5pt;font-family:"Helvetica","sans-serif";color:#1155cc;background:#ffee94">orcid</span><span style="font-size:11.5pt;font-family:"Helvetica","sans-serif";color:#1155cc">.org/0000-0002-0658-0814</span></a><br>
Information Manager, Baltimore Ecosystem Study<br>
Cary Institute of Ecosystem Studies<br>
Box AB; Route 44A<br>
Millbrook, NY 12545-0129<br>
P: <a href="tel:845%2F677%2F7600%20Extension%20103" target="_blank">845/677/7600 Extension 103</a><br>
F: <a href="tel:845%2F677%2F5976" target="_blank">845/677/5976</a><br>
E: <a href="mailto:WalshJ@caryinstitute.org" target="_blank">WalshJ@caryinstitute.org</a><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
______________________________<wbr>_________________<br>
Long Term Ecological Research Network<br>
im mailing list<br>
<a href="mailto:im@lternet.edu" target="_blank">im@lternet.edu</a><br>
<br>
<u></u><u></u></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div></div></div>
</div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr">Jonathan Walsh</div><div dir="ltr"><a href="http://orcid.org/0000-0002-0658-0814" style="color:rgb(17,85,204);font-size:12.8px" target="_blank"><span><span style="font-size:11.5pt;font-family:Helvetica,sans-serif;background:rgb(255,238,148)"><span>orcid</span></span></span><span style="font-size:11.5pt;font-family:Helvetica,sans-serif">.org/0000-0002-0658-0814</span></a><br>Information Manager, Baltimore Ecosystem Study<br>
Cary Institute of Ecosystem Studies<br>
Box AB; Route 44A<br>
Millbrook, NY 12545-0129<br>
P: <a href="tel:845%2F677%2F7600%20Extension%20103" value="+18456777600" target="_blank">845/677/7600 Extension 103</a><br>
F: <a href="tel:845%2F677%2F5976" value="+18456775976" target="_blank">845/677/5976</a><br>
E: <a href="mailto:WalshJ@caryinstitute.org" target="_blank">WalshJ@caryinstitute.org</a></div></div></div></div></div></div></div></div></div></div></div></div>
</div>