Blog Archives

Amazon ECS at The Climate Corporation: Using Amazon ECR and Multiple Accounts for Isolated Regression Testing

This is a post from Nathan Mehl, a Senior Staff Engineer, on the Operations Engineering team at The Climate Corporation. — Like many large users of AWS, The Climate Corporation uses multiple AWS accounts in order to provide strict isolation

Posted in Engineering

TCC Field Experiments on Sub-Field Variability of Weather

Hi, I’m Adam Pasch, Weather Data Manager and a member of the Meteorology team at TCC.  I’ll be presenting a poster at American Meteorological Society (AMS) Annual Meeting next week talking about some initial analyses of data from TCC field

Tagged with: , ,
Posted in Engineering, Science

Big Data, Cloud and NOAA CRADA’s with AWS

New Way

Hi, I’m Lak and I lead the Meteorology team at TCC.  I’ll be on a panel at the American Meteorological Society Annual Meeting next week talking about the NOAA Big Data cooperative research and development agreement with Amazon Web Services

Tagged with: , , ,
Posted in Engineering, Science

How to read and display Nexrad on AWS using Python

Amazon Web Services (AWS) is making NEXRAD data freely available on Amazon S3 as part of the NOAA Big Data Project. In this Python notebook, I will step you through being able to read and display this data from your

Tagged with: , ,
Posted in Engineering

Squeedo: Simplified Amazon SQS processing in Clojure

Inspiration At The Climate Corporation, we use various forms of data and and Clojure based computations to help farmers make decisions about their operations.  In order to do this quickly and efficiently, we distribute the processing across machines, often using

Tagged with: , ,
Posted in Engineering, Software Engineering

xray + dask: Out-of-core, labeled arrays in Python

Xray provides labeled, multi-dimensional arrays. Dask provides a system for parallel computing. Together, they allow for easy analysis of scientific datasets that don’t fit into memory. An introduction to xray To make it easier to work with scientific data in

Posted in Engineering

Numba vs Cython: How to Choose

My name is Stephan, and I’m a scientist on the Climatology team at The Climate Corporation. We make extensive use of Python to build statistical weather models, and sometimes we need our code to be fast. Here’s how I choose

Tagged with: , , , ,
Posted in Engineering