In this blog I share some experiences implementing a set of AWS Lambda Functions in both Node.js 6.10 and Java 8. I’ll begin with some initial observations before moving on to focus on performance and cost.
The AWS Lambda Functions
Earlier on this year I created a set of Node.js AWS Lambda Functions as part of my Smart Security Camera project. This project added image analysis to a Pi Zero web camera, allowing it to distinguish between real threats (i.e. burglars) and false alarms (i.e. the neighbours cat).
Once the original Node.js version of this project was completed I decided to recreate the same AWS Lambda Functions with Java, with the aim of comparing how Node.js and Java work on AWS Lambda.
The AWS Lambda Functions are as follows:
Java AWS Lambda Function
Node.js AWS Lambda Function
AWS Service Calls
Triggered when a new image is uploaded to a given s3 upload folder.
Uses AWS Rekognition to generate a list of labels describing each uploaded picture.
Evaluates labels to find out if an alarm email should be sent.
Sends an alarm email via AWS SES and JavaMail when the smart security camera detects a person.
Moves the processed image to the correct archive location in S3.
Table 1 : Lambda Function Overview
From a functional perspective the two sets of Lambda Functions are identical. Some, but not all, make calls to one of the following AWS Services:
- AWS S3 - Copy and deleting files on S3. Monitoring S3 for new Image Uploads.
- AWS Rekognition - Image Analysis of a image uploaded to S3.
- AWS SES - Sending an email with an Image attachment.
Differing Memory Requirements
When creating Lambda Functions you can choose how much memory the functions are allocated at runtime. The more memory you allocate, the higher the cost per CPU second.
My Node.js AWS Lambda Functions were happy with the minimum memory allowance of 128MB. However my Java based Functions tended to terminate with OutOfMemoryError exceptions unless they were allocated at least 192MB and preferably 256MB. This resulted in a higher base cost per CPU second when using Java.
We'll look more closely at the relative costs of using Java vs. Node.js later on in this blog entry.
Size of Created Artefacts
Another difference between Node.js and Java was the size of the created artefacts.
|Java AWS Lambda Function||Java Function Code Size||Node.js Code Size||Node.js AWS Lambda Function|
Table 2 : Relative sizes of Node.js and Java artefacts
As you can see the Node.js artefacts are substantially smaller than their Java counterparts. There are a couple of reasons for this.
Reason 1: The AWS Toolkit for Eclipse
When creating the Java version of my Smart Security Camera project I used the AWS Toolkit for Eclipse, which provides support for creating, packaging and uploading AWS Lambda Functions. Functions are packaged into ZIP files which also include a set of dependancy JAR files (including the AWS SDK).
Unfortunately the AWS Toolkit isn't selective about which JAR's it adds. A quick analysis of a deployed ZIP file revealed that it included a set of AWS Client JAR's that were not required for that specific Lambda Function to run.
Amending the Maven POM file to remove these unneeded JAR's helped reduce the size of the Java Functions by up to 2MB.
Reason 2: Writing Node.js Code Directly in the Browser
If your Node.js based AWS Lambda Function doesn't require any 3rd party libraries, you can choose to write your code directly in the browser and avoid the packaging and upload steps. If you do this, the AWS SDK is implicitly available (no need to upload it).
In the above table, the four "smallest" functions were all written in this way, which explains their small size.
The "largest" Node based AWS Lambda Function was packaged as a ZIP file due to it's dependancy on the Nodemailer library.
Comparing Performance and Cost
AWS Lambda Pricing Model
Simply put, AWS Lambda is billed by Requests and Duration.
The first million requests per month are free. After this you will be charged $0.20 per 1 million requests thereafter.
Duration is calculated from the time your code begins executing until it returns or otherwise terminates, rounded up to the nearest 100ms.
Both price and the free usage limit depend on the amount of memory you allocate to your function, as shown in the table below.
|Memory (MB)||Free tier seconds per month||Price per 100ms ($)|
Table 3 : Duration pricing examples
You can find more detailed information about AWS Lambda pricing at https://aws.amazon.com/lambda/pricing.
Testing AWS Lambda Performance
I created a test battery that would simulate approximately 1500 alerts through my Smart Security Camera. This would result in all the above Lambda Functions being called - once for each alert.
To simulate the behaviour of my camera, alerts were triggered in batches of 30, with a 60 second pause between them.
As mentioned earlier in this blog, the Java based AWS Lambda Functions required 256MB memory to ensure that no OutOfMemoryException errors were thrown. To be fair I therefore allocated the same amount of memory to the Node.js Functions.
Before running the test batteries the Lambda Functions were allowed to be inactive for a period of 90 minutes.
Finally I used a dashboard that I created via AWS CloudWatch to view and analyse the results.
The initial results showed a marked difference in average performance between Java and Node.js.
|Java AWS Lambda Function||Java Average Duration||Node.js Average Duration||Node.js AWS Lambda Function|
|s3-trigger-image-processing||970 ms||419 ms||s3-trigger-image-processing|
|rekognition-image-assessment||2.25 sec||1.72 sec||rekognition-image-assessment|
|rekognition-evaluate-labels||6.57 ms||1.61 ms||rekognition-evaluate-labels|
|ses-send-notification||3.04 sec||996 ms||nodemailer-send-notification|
|s3-archive-image||1.05 sec||364 ms||s3-archive-image|
Table 4 : Duration comparison based on 256MB memory allocation
The following two charts, generated by AWS CloudWatch, show how the average duration varied over the course of the test. Note that the vertical axis (representing duration) has different scales for Node.js and Java.
Figure 1 : Chart showing average duration for Java functions during the test. The vertical axis represents the average function execution duration in milliseconds, while the horizontal axis represents time
Figure 2 : Chart showing average duration for Node.js functions during the test. The vertical axis represents the average function execution duration in milliseconds, while the horizontal axis represents time
Observation 1 : Java costs more than Node.js on AWS Lambda
Based on the results of this test (and pricing information from AWS) I was able to extrapolate a total duration cost for these Lambda Functions processing 1,500,000 alerts in a month. When making this extrapolation I did not factor in the Free Tier allowance.
Table 5 : Cost extrapolation
While these figures look pretty definitive, lets remember that they are based on a test with only 1500 iterations. The Java figures will therefore be strongly affected by the time it takes to warm up the JVM.
Observation 2 : Cold starts with the Java container take longer
AWS Lambda works by spinning up runtime containers on demand. While there is increasing demand, more runtime containers will be provisioned. When demand shrinks, idle runtime containers are removed.
The process of spinning up a new runtime container is known as a cold start. If you take a look back at figures 1 and 2 you can see these cold starts at the beginning of the test, where the average duration used to execute the functions is longer. Once the containers are warmed up the average execution times drop down and plateau.
Figures 1 and 2 show that both Node.js and Java require cold starts and that both are nicely warmed up after 5 minutes or so. But the cost (i.e. average duration) of the Java cold starts is much higher for Java than for Node.js.
The difference in size between the Node.js and Java artefacts are most likely also a factor in the amount of time to cold start the respective containers.
Observation 3 : The AWS Lambda JVM is effectively a black box
You can tweak the amount of memory available to the JVM, but that seems to be about it. It may possible that further tweaking (for example around Garbage Collection) may improve the Java results from this test.
Observation 4 : A good use case for Project JigSaw and Java 9
Project JigSaw refers to the modularisation of the JDK that is coming in Java 9. This will reduce the JDK's footprint and hopefully also help improve JVM performance. With a Java 9 release date of September 2017 it could take a while before a modular JDK appears on AWS Lambda.
Observation 5 : Although the two sets of Lambda Functions are functionally identical, the underlying implementations are very different
My two Lambda Function stacks use different third party implementations to solve common problems such as:
- Sending Emails: For my Java implementation I use JavaMail and for my Node.js implementation I use NodeMailer.
What this all means is that both the size and performance of my AWS Lambda Functions will be influenced by the implementations and sub-dependancies of these third party libraries.
Observation 6 : Calls to the AWS SDK slow things down
For both Node.js and Java the fastest Function calls were those that did not make a subsequent call to other AWS services:
- rekognition-evaluate-labels: average 6.57 ms on Java, 1.61 ms on Node.js.
- s3-trigger-image-processing: average 970 ms on Java, 419 ms on Node.js.
This is not really surprising as functions making calls to other AWS services (i.e. Recognition, SES or S3) will need to wait for the underlying function call to complete, therefore upping their execution time. The lesson here would therefore be to avoid synchronous calls where possible.
This article sums up my experiences with measuring performance of Node.js 6.10 and Java 8 on AWS Lambda.
That Node.js outperformed Java can likely be attributed to the following:
- The relatively larger size of the Java ZIP files.
- A longer cold start for Java containers in AWS Lambda.
- A lack of options for tuning the JVM in AWS Lambda.
- Fundamental differences in the implementations of the Java and Node.js AWS Lambda Functions.
As AWS Lambda is partially billed by execution duration, one can also argue that Java AWS Lambda Functions are more expensive than Node.js equivalents. However this argument is only relevant if you plan on exceeding the free tier limits.
Does all this mean that Java is a no-go on AWS Lambda? Not at all. Strict typing and the ecosystem around Java make it a very effective platform for creating software. Eclipse plugins and Maven archetypes exist to help one quickly get up and running with Java based AWS Lambda Functions.
Finally thanks for reading this blogpost! If you have any comments or questions, please share them in the comments below!
More information about the Smart Security Camera Project is available via the following links: