-->

TEST

Thursday 29 March 2012

1000 Genomes Project and AWS

Data from the 1000 Genomes project - The world's largest set of data on human genetic variation produced by the international 1000 Genomes Project — is now publicly available on the Amazon Web Services (AWS) cloud (http://aws.amazon.com/1000genomes/). See the NIH press realease for more information: (http://1.usa.gov/Hnt1f7). 1000 genomes data may also be downloaded from the NCBI though ftp (http://bit.ly/Hlj3wM) or through the Aspera protocol (http://1.usa.gov/d9ON7X)

1000 Genomes Project and AWS

The 1000 Genomes Project is an international research effort coordinated by a consortium of 75 companies and organizations to establish the most detailed catalogue of human genetic variation. The project has grown to 200 terabytes of genomic data including DNA sequenced from more than 1,700 individuals that researchers can now access on AWS for use in disease research. The 1000 Genomes Project aims to include the genomes of more than 2,662 individuals from 26 populations around the world, and the NIH will continue to add the remaining genome samples to the data collection this year.
The dataset containing the full genomic sequence of 1,700 individuals is now available to all via Amazon S3. The data can be found at: s3.amazonaws.com/1000genomes

Accessing 1000 Genomes Data

AWS is making the 1000 Genomes Project data publicly available to the community free of charge. Public Data Sets on AWS provide a centralized repository of public data hosted on Amazon Simple Storage Service (Amazon S3). The data can be seamlessly accessed from AWS services such Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Elastic MapReduce (Amazon EMR), which provide organizations with the highly scalable compute resources needed to take advantage of these large data collections. AWS is storing the public data sets at no charge to the community. Researchers pay only for the additional AWS resources they need for further processing or analysis of the data. Learn more about Public Data Sets on AWS.
All 200 TB of the latest 1000 Genomes Project data is available in a publicly available Amazon S3 bucket.
You can access the data via simple HTTP requests, or take advantage of the AWS SDKs in languages such as Ruby, Java, Python, .NET and PHP.

Analyzing 1000 Genomes Data

Researchers can use the Amazon EC2 utility computing service to dive into this data without the usual capital investment required to work with data at this scale. AWS also provides a number of orchestration and automation services to help teams make their research available to others to remix and reuse.
Making the data available via a bucket in Amazon S3 also means that customers can crunch the information using Hadoop via Amazon Elastic MapReduce, and take advantage of the growing collection of tools for running bioinformatics job flows, such as CloudBurst and Crossbow.

Other Sources
The 1000 Genomes project data are also freely accessible through the 1000 Genomes website, and from each of the two institutions that work together as the project Data Coordination Centre (DCC).