AgiliWay, a software development outsourcing company (https://agiliway.com/), in partnership with a software distributor, Softprom (https://softprom.com/), has supplied to ROI4CIO and implemented AWS, which has improved website performance by 8 times. Implementation project timelines: 28.09.2017 – 18.10.2017: optimal server set-up chosen, optimal AWS services selected, complete transition to AWS performed, including the domain name. Issue: Services used by the website prior to AWS implementation did not support operations with “heavy” functionality, such as processing large volumes of data. For this reason when the number of website users reached a certain figure, CPU and RAM indicators of the server used to rise to 100% and the website crashed. Solution: Thanks to AWS server and services the number of concurrent users of the site has increased several times, whereas the workload on the site and database is half as much as before. The website performance has improved by 8 times, however its cost of ownership has grown only twofold. ROI - 800% Issue:

• "The team has come up against the problem of insufficient website performance. When during testing the number of concurrent users reached a certain figure, the response time delay exceeded 10 seconds. As workload increased, the database failed to support it and the service “refused” to work,” - Oleg Pitsyk, IT Systems’ Architect with ROI4CIO, Agiliway.

Solution: Before the decision was taken to implement AWS for the ROI4CIO project, it was tested whether another cloud-based solution can potentially be used. Comparison of the testing results has conclusively proven superiority of AWS. Softprom in cooperation with Agiliway has developed the optimal solution enabling to enhance the site performance. Representatives of Agiliway, acting as developers and architects of ROI4CIO systems, were in charge of application server and database set-up. Softprom, a supplier of AWS platforms, acted as a consultant with regards to deploying AWS functionality. On top of that Softprom employees have configured the services of monitoring, notifications and AWS Lambda. The website domain was hosted in Amazon Route 53, highly available and scalable cloud Domain Name System service (DNS). Anti DDoS AWS Shield wall was deployed to ensure comprehensive protection from all known infrastructure attacks (levels 3 and 4). A virtual EC2 (C4 type) server was used to optimize workloads, which required high computing capacity, to configure automated scaling of resources and inbound loadbalancing. At website peak load times computing resources are automatically scaled, which allows to support virtually any workload. Amazon Relational Database Service was selected to enhance reliability and fail-safety, as well as to reduce administration spend. It performs the functions of hardware provisioning, database setup, patching and backups. ElastiCashe was also used to increase website throughput - it is a web service which simplifies rolling out and scaling functions in a cloud or in-memory store, as well as their management. AWS CloudWatch was used for extensive monitoring of applications and database. Provided relevant rules are in place, an administrator will be immediately notified by text or e-mail in case of any unexpected emergency. AWS SNS (simple notification service) was integrated in CloudWatch for messaging functions. Besides messaging in some circumstances serverless computations of AWS Lambda are triggered to fulfill automation functions. AWS S3 cloud storage automatically backs up the database and application server as scheduled.

• "Our efforts were mainly aimed at enhancing performance. However, a lot of attention was paid to fail-safety and backups. Quantitatively, we can probably also highlight Input/Output Operations Per Second (IOPS) of storage drive. We have used a high performance drive - Provisioned IOPS. Up to 40,000 IOPS can be provisioned for each DB instance", - Vlad Gavrilenko, CIO of Softprom.

• "The server was migrated very quickly, in two days literally. After that we were configuring the services, which has taken about a week," -  Vlad Gavrilenko, CIO Softprom.

Outcome: By implementing AWS the team has made it possible to increase the resource performance by eight times and at the same time to automate labour-intensive administration tasks. And it should be noted that cost of ownership has risen only twofold. After the query code was patched and all services have been implemented the site maintains stable and uninterrupted performance even at high load levels.

Established in 1958, the National Aeronautics and Space Administration (NASA) has been working around the world—and off of it—for almost 60 years, trying to answer some basic questions: What’s out there in space? How do we get there? What will we find? What can we learn there, or learn just by trying to get there, that will make life better here on Earth?
Exploring Space: No Rocket Science Degree Needed Have you ever looked up at night and wondered about the mysteries of space? Or marveled at the expansiveness of our galaxy? You can easily explore all this and more at the NASA Image and Video Library, which provides easy access to more than 140,000 still images, audio recordings, and videos—documenting NASA’s more than half a century of achievements in exploring the vast unknown. For NASA, providing the public with such easy access to the wonders of space has been a journey all its own. NASA began providing online access to photos, video, and audio in the early 2000’s, when media capture began to shift from analog and film to digital. Before long, each of NASA’s 10 field centers was making its imagery available online, including digitized versions of some older assets. Therein was the challenge: “With media in so many different places, you needed institutional knowledge of NASA to know where to look,” says Rodney Grubbs, imagery experts program manager at NASA. “If you wanted a video of the space shuttle launch, you had to go to the Kennedy Space Center website. If you wanted pictures from the Hubble Space Telescope, you went to the Goddard Space Flight Center website. With 10 different centers and dozens of distributed image collections, it took a lot of digging around to find what you wanted.” Early efforts to provide a one-stop shop consisted of essentially “scraping” content from the different sites, bringing it together in one place, and layering a search engine on top. “In large part, those initial efforts were unsuccessful because each center categorized its imagery in different ways,” says Grubbs. “As a result, we often had five to six copies of the same image, each described in different ways, which made searches difficult and delivered a poor user experience.” In 2011, NASA decided that the best approach to address this issue was to start over. By late 2014, all the necessary pieces for a second attempt were in place:

• The Imagery Experts Program had developed and published a common metadata standard, which all NASA’s centers had adopted.

• The Web Enterprise Service Technologies (WESTPrime) service contract, one of five agency-wide service contracts under NASA’s Enterprise Services program, provided a delivery vehicle for building and managing the new site.

• The Federal Risk and Authorization Management Program (FedRAMP), which provides a standardized approach to security assessment, authorization, and continuous monitoring for cloud products and services.

“We wanted to build our new solution in the cloud for two reasons,” says Grubbs. “By 2014, like with many government agencies, NASA was trying to get away from buying hardware and building data centers, which are expensive to build and manage. The cloud also provided the ability to scale with ease, as needed, paying for only the capacity we use instead of having to make a large up-front investment.”
Decades of NASA Achievements – All in One Place Development of the new NASA Image and Video Library was handled by the Web Services Office within NASA’s Enterprise Service and Integration Division. Technology selection, solution design, and implementation was managed by InfoZen, the WESTPrime contract service provider. As an Advanced Consulting Partner of the AWS Partner Network (APN), InfoZen chose to build the solution on Amazon Web Services (AWS). “Amazon was the largest cloud services provider, had a strong government cloud presence, and offered the most suitable cloud in terms of elasticity,” recalls Sandeep Shilawat, Cloud Program Manager at InfoZen. NASA formally launched its Image and Video Library in March 2017. Key features include:

• A user interface that automatically scales for PCs, tablets, and mobile phones across virtually every browser and operating system.

• A search interface that lets people easily find what they’re looking for, including the ability to choose from gallery view or list view and to narrow-down search results by media type and/or by year.

• The ability to easily download any media found on the site—or share it on Pinterest, Facebook, Twitter, or Google+.

• Access to the metadata associated with each asset, such as file size, file format, which center created the asset, and when it was created. When available, users can also view EXIF/camera data for still images such as exposure, shutter speed, and lens used.

• An application programming interface (API) for automated uploads of new content—including integration with NASA’s existing authentication mechanism.

Architecture The NASA Image and Video Library is a cloud-native solution, with the front-end web app separated from the backend API. It runs as immutable infrastructure in a fully automated environment, with all infrastructure defined in code to support continuous integration and continuous deployment (CI/CD). In building the solution, InfoZen took advantage of the following Amazon Web Services:

• Amazon Elastic Compute Cloud (Amazon EC2), which provides secure, resizable compute capacity in the cloud. This enables NASA to scale up under load and scale down during periods of inactivity to save money, and pay for only what it uses.

• Elastic Load Balancing (ELB), which is used to distribute incoming traffic across multiple Amazon EC2 instances, as required to achieve redundancy and fault-tolerance.

• Amazon Simple Storage Service (Amazon S3), which supports object storage for incoming (uploaded) media, metadata, and published assets.

• Amazon Simple Queue Service (SQS), which is used to decouple incoming jobs from pipeline processes.

• Amazon Relational Database Service (Amazon RDS), which is used for automatic synchronization and failover.

• Amazon DynamoDB, a fast and flexible NoSQL database service, which is used to track incoming jobs, published assets, and users.

• Amazon Elastic Transcoder, which is used to transcode audio and video to various resolutions.

• Amazon CloudSearch, which is used to support searching by free text or fields.

• Amazon Simple Notification Service (SNS), which is used to trigger the processing pipeline when new content is uploaded.

• AWS CloudFormation, which enables automated creation, updating, and destruction of AWS resources. InfoZen also used the Troposphere library, which enables the creation of objects via AWS CloudFormation using Python instead of hand-coded JSON—each object representing one AWS resource such as an instance, an Elastic IP (EIP) address, or a security group.

• Amazon CloudWatch, which provides a monitoring service for AWS cloud resources and the applications running on AWS.

An Image and Video Library for the Future Through its use of AWS, with support from InfoZen, NASA is making its vast wealth of pictures, videos, and audio files—previously in some 60 “collections” across NASA’s 10 centers—easily discoverable in one centralized location, delivering these benefits:

• Easy Access to the Wonders of Space. The Image and Video Library automatically optimizes the user experience for each user’s particular device. It is also fully compliant with Section 508 of the Rehabilitation Act, which requires federal agencies to make their technology solutions accessible to people with disabilities. Captions can be turned on or off for videos played on the site, and text-based caption files can be downloaded for any video.

• Built-in Scalability. All components of the NASA Image and Video Library are built to scale on demand, as needed to handle usage spikes. “On-demand scalability will be invaluable for events such as the solar eclipse that’s happening later this summer—both as we upload new media and as the public comes to view that content,” says Bryan Walls, Imagery Experts Deputy Program Manager at NASA.

• Good Use of Taxpayer Dollars. By building its Image and Video Library in the cloud, NASA avoided the costs associated with deploying and maintaining server and storage hardware in-house. Instead, the agency can simply pay for the AWS resources it uses at any given time.

While NASA’s new Image and Video Library delivers a wealth of new convenience and capabilities, for people like Grubbs and Walls, it’s just the beginning. “We now have an agile, scalable foundation on which to do all kinds of amazing things,” says Walls. “Much like with the exploration of space, we’re just starting to imagine all that we can do with it.”

"With AWS Lambda, our various engineering teams can tap into a parallel data stream to create microservices independently from the main analytics application. It helps us get new services to our customers faster. For a startup, faster time to market is key", Mohit Dilawari, Director of Engineering

The Challenge

• Supports pipeline with billions of data points uploaded every day from different mobile applications running Localytics analytics software.

• Engineering team needed to access subsets of data for creating new services, but this led to additional capacity planning, utilization monitoring, and infrastructure management.

• Platform team wanted to enable self-service for engineering teams.

Before using Amazon Kinesis and Amazon Lambda, the main analytics processing service for Localytics had to be updated when a microservice was added.

The Solution

• Uses AWS to send about 100 billion data points monthly through Elastic Load Balancing to Amazon Simple Queue Service, then to Amazon Elastic Compute Cloud, and finally into an Amazon Kinesis stream.

• For each new feature of the marketing software, a new microservice using AWS Lambda is created to access the Amazon Kinesis data stream. Each microservice can access the data stream in parallel with others.

With Amazon Kinesis and Amazon Lambda deployed, Localytics puts a subset of data into a Kinesis stream, which different microservices teams can use to build their own Lambda microservices without needing to notify or consult the analytics team.

The Benefits

• Decouples product engineering efforts from the platform analytics pipeline, enabling creation of new microservices to access data stream without the need to be bundled with the main analytics application.

• Eliminates the need to provision and manage infrastructure to run each microservice .

• Lambda automatically scales up and down with load, processing tens of billions of data points monthly.

• Speeds time to market for new customer services, since each feature is a new microservice that can run and scale independently of every other microservice.