When NRK and DNT decided to upgrade their outdoor trekking planner, they decided to run it in the cloud using Windows Azure. In this blogpost I will explain the customer's requirements, the solution and what benefits we experienced using cloud computing when creating the new web portal (currently at beta.ut.no).
NRK (Norwegian Broadcasting Corporation) is the government-owned public broadcasting company, and the largest media company in Norway. They provide media services through web, radio and TV. DNT (Norwegian Trekking Association) is Norway's largest outdoor activities organization, promoting trekking across Norway's diverse nature with content about trekking, lodging, points of interest and more. The portal UT.no is a cooperative venture between NRK and DNT that provide users with a joined information portal leveraging the quality of NRK's editorial content with DNT's large community-driven content archive. One of the portal's main goals is to get its users to go outdoors and enjoy nature. It has about 10 000 - 15 000 unique visitors per day and has a companion app for Android and iOS.
The old on-premise PHP-driven portal had entered an unmaintainable state where more time was being spent on bug-fixing than development, and needed to be re-implemented with a fresh approach. The new portal was to be implemented with Extreme Programming (XP) principles, be mobile-first designed and focus on these key features:
- Give users relevant content
- Expose an API for apps, map and other consumers
The app was also going to be redesigned to use the new API. Earlier versions had used XML-scraping and other techniques to deliver content to the app, but the new version should utilize an API instead and should not have to go through a new publish cycle in the App Stores each time they wanted to change or redesign content. The portal also contained a Leaflet-powered interactive map which users used to discover content geographically, and it should also be redesigned to use the new API. The map had specific needs for querying content inside a geographical bounding box (north, south, east, west) and content along a trekking path. A lot of the content on the portal was season- and event-driven, so there was a need to be dynamic in both design and content. Before events or holidays NRK may broadcast promotions for the portal and app on TV and nrk.no, so the site and its services should also respond well to traffic-spikes.
NRK had been an early adopter of Windows Azure and having several sites and services already running in Windows Azure, placing the new portal in Azure was a strong desire from the management. The project had focus on low cost, so in terms of cost-saving measures they wanted to:
- Minimize usage of dedicated on-premise hardware and services (hosting, supporting services)
- Minimize usage of internal resources (operational support) and minimize their workload
- Maximize usage of Azure resources
The vision for the new portal was to implement a content-by-search strategy, giving the users added value when browsing content. The key search factors were:
- User's location (browser location or GPS-location via the app)
- User's preferences and search history
- Content's relation to other content (relevance, popularity, proximity to user's location and to each other)
The search engine requirements were that it needed to support queries that were geographic-, score- and index-based. It should also be flexible in terms of different methods data is collected and it should be possible to roll out and scale in Azure without requiring too much detail knowledge about the technical implementation.
The portal was implemented in ASP.Net MVC & Web API and uses ElasticSearch as the search engine powering the site. ElasticSearch was chosen because it could solve the search-requirements of the portal in one complete solution. The previous portal used GeoServer in Amazon EC2 in addition to its own search, but with ElasticSearch supporting geographic querying on geopoints (in addition to score-based and index-based querying) and being open-source, it became a more attractive option. There was also interest in using ElasticSearch in other teams inside the organization which makes internal resource planning across the projects easier for the management. ElasticSearch indexes data via data "rivers" connected to DNT's REST API via a changelog. This ensures that the portal and API has the latest content available as soon as the community adds new content via DNT or when NRK publishes new articles.
- SQL Azure for storing site-settings
- Virtual Network for containing provisioned roles (VMs) in subnets, allowing for Azure Point-to-Site VPN.
- Cloud services for each deployment environment
- Web roles for the ASP.Net website
- Worker roles for ElasticSearch search engine
- Logging and diagnostics in Azure Table Storage
- Azure AD for handling users and enabling single-sign on with DNT
When the decision was made to move from on-premise to cloud there was certain cost-savings and improvements made up-front, such as:
- Old on-premise web and database servers can be re-tooled for other uses
- On-premise network and load-balancers will get less traffic
- Fewer internal operational resources needed to monitor the services
- Higher availability and scaling on-demand
But beyond this, we saw that:
- having all web and worker roles in a Virtual Network allows the developers quicker and more efficient access to debugging than before with previous portal
- time was saved when the team could scale-out provisioned roles without involving operational resources
- time was saved when deploying through the deployment environments as the team could now trigger builds and deployments by themselves
- upscaling/downscaling speed was improved significantly as one would have to contact internal operational resources to provision servers previously
- deployment to the development cloud service became a good motivational factor, as nightly builds and deployment (if all tests passed) gave developers a chance to see their features working in Azure with Azure resources leveraged
- it was less complicated to specify requirements for Staging, QA and Production as this was now just a matter of configuration transforming and targeted deploys, whereas previously one would have to involve operational support for deployment through the environments
- internal operational resources can be more effective in managing deployed services as all of the sites and services under their control are listed in the same management portal
- scaling ElasticSearch via configuration became "fun" as the team could see that more nodes became available when the instance count on the worker role was increased, and equally decreased as they scaled down. Similar scaling efforts had not been possible.
The team were able to have more ownership of their own product with the control of deployments and provisioned resources. Operational resources were still involved throughout but at a lesser extent. The team also saw quicker "time-to-market" on new features added to the portal and each design iteration became more efficient as the team settled into their new work-cycle. In terms of provisioned hardware the project as a whole probably uses more hardware than before (an additional integration-testing environment was also added), but they are scaled better to fit project's actual needs than before so the hardware-cost still remains lower than the on-premise hardware.
By moving to Windows Azure and using technologies such as ASP.Net MVC & Web API with ElasticSearch, the new portal can be run more efficiently when utilizing provisioned resources rather than with static on-premise hardware, and be more agile in terms of scaling for varying demand and traffic spikes. The team saves cost by reducing the need to involve internal resources for deployment planning and by automating deployment. Overall the project and the new portal is running more efficiently and can concentrate the cost-savings on adding new features and giving business value to its users.