amazon-web-services,amazon-ec2,cloud,server,amazon , AWS Instance Type For 3,000+/Second Outgoing HTTP Requests?

AWS Instance Type For 3,000+/Second Outgoing HTTP Requests?


Tag: amazon-web-services,amazon-ec2,cloud,server,amazon

I'm currently scaling up an application on AWS, we send thousands of outgoing HTTP requests per second to 3rd party API's and services. Are there any EC2 instances optimized for this type of work?


When work in aws, you need jump out of the idea as traditional way, because you needn't worry about if the instance is not suitable, waste money, etc.

Estimate how big traffic go out and you can start with a reasonable instance type, for example, t2.medium.

After running your application and keep monitoring the performance, if it is not good enough, just stop the instance, change the instance type, and start it again.

Here is the official document for Resizing Your Instance

That's all you need do, don't worry it from beginning.


Polling Continuously from a SQS queue on AWS

I have a java class that connects to an SQS queue and I would like it to respond to messages that are sent to the SQS queue. Is that possible without running the java class continuously, sending receiveMessageRequests?

how to use AWS cognito with custom authentication to create temporary s3 upload security token

So I'm a bit confused by the Amazon documentation on Cognito concerning one of their stated use cases: "use your own identity system... allowing your apps to save data to the AWS cloud". In my case I want to give them aws tokens to upload directly to s3 from the...

Deleting Data from DynamoDb Table automatically

Is there any kind of life retention period concept in DynamoDB. I mean is there any way such that data inside a table will be deleted after some time like we can set some retention period in S3. Thanks,...

Amazon EC2 Storage lacks

I have launched Amazon EC2 instance of "m3.large" type. According to this page, m3.large should have 2vCPUs, 7.5GiB Memory and 1x32GB SSD Storage. But df -ah returns following results. It seems that the instance lacks the volume. Filesystem Size Used Avail Use% Mounted on /dev/xvda1 7.9G 797M 6.7G 11% /...

AWS Beanstalk - Passenger Standalone not serving web pages after Rails 4.2.1 migration

My Rails 3.2.21 app was running fine on AWS Beanstalk under Passenger Standalone 4.0.53. I migrated the app to Rails 4.2.1 and got it passing all tests on my local development machine (Ubuntu, WEBrick). I deployed it to Beanstalk (aws.push), the deploy succeeds (copied from /ondeck to /current) and: nothing....

Eclipse not compiling because of ClassNotFoundException

After following an AWS tutorial for Eclipse, my code no longer compiles and runs. I decided to undo what the tutorial told me, so I may have changed some settings that I forgot to unchanged but I really cannot find the root of my problem. Eclipse seems to be back...

Amazon DynamoDB Mapper - limits to batch operations

I am trying to write a huge number of records into a dynamoDB and I would like to know what is the correct way of doing that. Currently, I am using the DynamoDBMapper to do the job in a one batchWrite operation but after reading the documentation, I am not...

PHP: Secure a Rest Service with a Token mixed with Timestamp

I have a rest service that my website calls it and I want to secure it from calling outside of my website as much as possible. I want to create a token mixed with timestamp, so the user can only call the service in 10 minutes (for example) with the...

Xcode + AWS Integration Apple Mach-O Linker Error

I have a very simple Xcode project that I started and am now trying to integrate the Amazon Web Service (AWS) SDK into my project. I followed the instructions posted on their instruction page and everything looks good EXCEPT that I've got the following output... duplicate symbol _OBJC_CLASS_$_XMLDictionaryParser in: /Volumes/Macintosh...

Installing Python 3 Docker Ubuntu error command 'x86_64-linux-gnu-gcc

I'm trying to create a dockerfile that uses Python 3. FROM ubuntu:14.04 RUN apt-get update RUN apt-get install -y python3 python3-dev python-pip RUN apt-get install -y libxml2-dev libxslt1-dev libpq-dev libjpeg-dev libfreetype6-dev zlib1g-dev RUN cd /var/projects/apps && pip install -r requirements.txt I get the error fatal error: Python.h: No such file...

Use Reserved instance and autoscaling group

I would like to know if it would possible to create an architecture with both reserved instance (RI) and auto-scaling group to serve web pages. The idea would be to have one RI serving 24/7 and launching on demand instances in an auto-scaling group when the CPU of the RI...

DynamoDB Conditional Check Fail Monitoring

I have used dynamodb-session to set DynamoDB for Session state provider. In my ASP.NET_SessionState table in DynamoDb There is "Conditional CheckFailed " monitoring. My question is: In what condition these exceptions happen and how can I reduce them?...

How to route traffic by proximity from Route 53 to closest NGINX server?

I'm trying to set up a web server stack in the following way: Use Route 53 for my DNS Serve static content with Cloudflare from S3 buckets Route API calls to nearest NGINX server that sits in front of some Nodejs servers. So all static content is done easily enough...

AWS Elastic beanstalk scale triggering

I set the following parameters in my elastic beanstalk environment: Do you think this settings are reasonable? I didn't understand the breach duration parameter. What does it means? is 5 minutes is reasonable? Thanks...

cloudsearch query to boost exact match on range

In a cloudsearch structured query. I have a couple of fields I am searching on. On field one, the user selects "2" On field two the user selects "1" I am wanting to run this as a range query, so that the results that are returned are -1 to +1...

Configure Dockerfile to set AWS configurations

I've just started looking at Docker. I have a node app that resizes and image and then sends an SQS message to aws when finished. I have managed to create a docker image of my app, copying it from my local machine, but run into the issue that I can't...

Is there a way to cluster different machines running Tomcat instances

I have setup multiple tomcat instances in a single machine to launch more threads, however, if I have to make use of 10 EC2 instances with Tomcat installed as a cluster, how would I achieve this. Any good resources to look at?

AWS RDS on Eclipse

I know this question seems a repeat, but it's not and I have tried all the solutions I could find PROBLEM: I am running an AWS RDS instance for the database. It works fine when I connect to it using AWS EC2 instance(uses linux) but when I try it with...

Spring Boot War file gets 404 on ec2

I put together a simple Eclipse Spring Boot Rest program. Just a couple of endpoints that return some stings. I am building it with the Eclipse Gradle plug-in. No problems, it runs fine in the Eclipse provided Tomcat instance. I also ran it on a native windows Tomcat. I deployed...

How to set a variable using dynamic inventory using Ansible

I am looking for method to set a variable in ansible playbook using inventory information received from dynamic inventory. For example if we have a sample playbook like --- - hosts: localhost connection: local tasks: - set_fact: rds_hostname="{{ rds_mysql }}" #set rds endpoint from - debug: var=rds_hostname I am...

How to transfer files from iPhone to EC2 instance or EBS?

I am trying to create an iOS app, which will transfer the files from an iPhone to a server, process them there, and return the result to the app instantly. I have noticed that AWS offers an SDK to transfer files from iOS app to S3, but not to EC2...

Secure file upload directly to s3 or server to s3 (from iOS app) [closed]

I need to upload sensitive images to s3 from an iOS app. I'm wondering which option is better: Upload to my server first, then upload to s3. Upload to s3 directly, then upload metadata to my server. ...

Loop through list of AWS-instances shows only first item

I am working on a simple customer frontend for AWS. I want a list of all the users machines for start/stopping the EC2s. While the logic works I can only show the first of the machines in my view. I guess it's related to the AWS APIs pageable response format,...

How to change the IP address of Amazon EC2 instance using boto library

How can I assign a new IP address (or Elastic IP) to an already existing AWS EC2 instance using boto library.

access tomcat deployed application (aws) by domain name (

I am new to aws hosting, please help me out in pointing the domain name to tomcat hosted application. During configuration following steps are taken- 1- I have created ec2 ubuntu instance. Configured with java, tomcat. Deployed my war files on tomcat. I am able to access the application successfully....

Gradle Compilation Errors when adding 'com.amazonaws:aws-java-sdk:1.10.1'

I am running Android Studio and my app has been running perfect until I do the following. When I add "compile 'com.amazonaws:aws-java-sdk:1.10.1'" to my gradle file, I get the following warnings when I do a project re-build. Warning:Dependency commons-logging:commons-logging:1.1.3 is ignored for debug as it may be conflicting with the...

Using Java web service on Amazon cloud

I want to make a web service in java which will take arguments and do processing and return a json response. I am not been able to figure out how to deploy this service on amazon ie (on ec2 or some where else) . what will be the url to...

Why is this python boto S3 multipart upload code not working?

I am trying to upload a 10 GB file to AWS S3, and someone said to use S3 Multipart Upload, so I stumbled upon someone's github gist: import os import sys import glob import subprocess import contextlib import functools import multiprocessing from multiprocessing.pool import IMapIterator from optparse import OptionParser from...

Keep config file secure using github and Elastic Beanstalk?

I am using github (public) to keep track of my web app and about to deploy it to Elastic Beanstalk. Is there a good way to keep my config file secure which has RDS username/password? I have to add the file to git in order to push it to Elastic...

AWS Beanstalk autoscale user files

I have setup AWS Beanstalk instance where a server app is deployed. In the backend users can change files in images/ directory. But when autoscaling the instances, the user files are not mirrored. How to solve this requirement? Can I setup AWS Ec2 to create new AMI each night based...

ajax GET request times out for URL when browser and CURL work

I see one similar question but it does not have an accepted response. The following ajax request times out. But GET request on the same URL using browser or curl work fine. Note this is a cross domain AJAX since the code sits on a different server and URL is...

AWS Kinesis - data source on a third party server

New to AWS Kinesis. We're trying to evaluate whether it makes sense or even possible to place events captured in a log file which is located on a third party server into AWS Kinesis stream, given that we only have a VPN access to this server where the log file...

How to find Unused Security Groups of all AWS Security Groups?

How to find all the used security groups attached with all the aws resources using Boto? Currently the following script which is giving only ec2 instances- sec_grps = ec2_conn.get_all_security_groups() for group in sec_grps: print group, " Instances attached ", group.instances() Is there any way to get all security groups which...

“undefined method 'value'” when looping

I am trying to loop through a list of tags returned from AWS API, but I'm getting "undefined method 'value'. I can provide further information if needed. This is my simple loop: @instances.each do |i| t = 0 while i.tags.any? do puts i.tags[t].value t += 1 end end ...

eb cli 3.0 is not putting my settings from my existing environment after connecting it

I created a customized Elastic Beanstalk environment from the web interface with configuration for VPC and other things. I now have a local repo that I want to connect to this created environment. I ran eb init and was able to spot my environment and selected it. I then ran...

Ruby aws-sk: undefined method

Trying to get ruby working with the ruby AWS sdk, but keep getting the following error: aws.rb:10:in <main>': undefined methodinstances' for # (NoMethodError) From other postings on StackOverflow, v2 of the aws-sdk changed from AWS to Aws. I also tried require 'aws-sdk' ec2 = access_key_id: 'mudd', secret_access_key: 'butt',...

How to configure aws CLI to s3 cp with anonymous user

I need to download files recursively from a s3 bucket. The s3 bucket lets anonymous access. How to list files and download them without providing AWS Access Key using an anonymous user? My command is: aws s3 cp s3://[email protected]/pavlo/text/tiny/rankings/uservisits uservisit --region us-east --recursive The aws compains that: Unable to locate...

How can I know the database url of AWS EC2 MySQL?

I would like to import gtfs files into mysql by using a tool from github, runing the follwoing command: gtfsdb-load --database_url <db url> <gtfs file | url> How can I get the database_url of mysql located in AWS EC2?...

Which is a better way: retrieve images from AWS S3 or download it and store locally in a temp folder to be displayed?

Problem: Retrieve image from S3 and load into UIButton. I'm currently doing my research on this issue and can't seem to make up my mind. Which is a better way to do it in terms of performance and security issue? Also, do I need to do caching or store these...

heroku pgbackups:url command is no longer working?

How do I download my dump directly from Amazon AWS S3 if heroku pgbackups:url b004 isn't working? Specifically, when I run this command it returns: ! Please add the pgbackups addon first via: ! heroku addons:add pgbackups And then when I run this command I get: ! No such add-on...

How do I SSH into EC2 with .pub?

When I create a new Elastic Beanstalk environment it asked me if wanted to create a new keypair. I say yes, and it created two file in my .ssh folder locally called app and Normally to ssh into an instance I use a app.pem file. i.e ssh -i app.pem...