[LTER-im-rep] Please consider completing short IM survey on storing, managing and delivering large data files!

[Linda A Powell]

Dear Information Managers,

It was so nice seeing everyone at the LTER ASM meeting last week!  Hope everyone had safe, uneventful travels home.


Due to a impromptu meeting with Saran (NSF), some of you weren’t able to attend the IM session on the “Challenges and Solutions Related to Storing, Managing and Delivering High Resolution Data” held on Tuesday, September 1st.  Jonathan, Eda and I are making an effort to put together three documents 1) a workgroup session report, 2) a paper on the workgroup discussion results and 3) a short article for DataBits.  I’ve included a list of questions on the session topic below and would really appreciate your input.  If you have the time and interest, please answer the questions and email them back to me by Tuesday, September 15th.  I apologize for any cross-postings!  Thank you in advance for taking the time out of your busy schedules to contribute to our effort.


Session Title:  Challenges and Solutions Related to Storing, Managing and Delivering High Resolution Data

Session Goals:

    a.  Share ideas and experiences with regards to storing, managing and delivering large data files.
    b.  Gather information on this topic to be used in an ‘All-sites’ document that discusses the current management methods used by LTER information managers.
    c.  Discuss possible collaborative solutions to the challenges such as having the ‘Data Center’ provide services for storing, managing and delivering these large data files.

Large Data Definition: By Large Data files , we mean file size and realize that what you may consider ‘large’ is relative to your existing resources.  It includes any data file that may be complex and/or large in physical size. Examples of these large data can include LIDAR, ASTER, LandSat, GIS, Acoustic, and streaming measurements.


Session Questions:


1.  How do Information Managers plan for future storage increases?  Are you part of the project planning phase from the start of a project planning on collecting large data files?  Do you require that PIs with new projects complete some kind of a project form that includes a section on what is being proposed in terms of data collection?

2.  Do you have an annual IM budget that allows for storage increases if necessary? Does the LTER site own their own servers and storage boxes or is the equipment leased from your institution?  What type of constraints or restrictions do you have with your systems?


For example,  the FCE LTER leases virtual servers from FIU and those servers do not have direct access to the internet.  This adds a lot of work to the system administration of the Oracle server because I can’t access the Oracle server patches directly using tools provided by the Oracle Enterprise Manager but instead have to download the patch to my workstation in the FCE office and migrate the files across the network and this can take the entire day sometimes (and there are lots of time-outs of the transfer!)

3.  How do you backup large data files?  Are backups local only or do you backup remotely, off-site?  Do you keep several versions of the large data files?

4.  Are you using ‘Cloud’ services and if so, which service?  How do you estimate costs (i.e. bandwidth, uploads/downloads) every month and is billing a problem for your institution because of the inconsistent monthly charges?  Any other constraints or problems when using the Cloud?

5.  How are your large data files being made accessible to researchers within the site, LTER network and to the broader community?


6.  Do you think the proposed ‘Data Center’ should handle the storing, managing and delivery of large data sets?  For example, offer a clip and ship type service where the user can choose a subset of a larger LIDAR data set and have it sent to them or even have the entire LIDAR data set mailed to them on an external hard drive.  The Data Center would also be responsible for backups of these large data sets.

7.  Any additional comments?


Kindest Regards,


Linda


Linda Powell
Information Manager
Florida Coastal Everglades LTER Program
OE 148, Florida International University
University Park
Miami, Florida 33199
Phone (Tallahassee, FL): 850-745-0381
Phone(Miami,FL): 305-856-0039 or  305-348-6054
Website: http://fcelter.fiu.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lternet.edu/pipermail/im-rep/attachments/20150908/10697d51/attachment.html>