Skip to main content

EC2 Instances Scheduling

This doc explains about the scheduling of aws ec2 instances. Below are the explanation.

EC2

  • Common AI central tranining server

Common AI central tranining server

In this we have setup scheduling for common ai central training servers.

  • Setup scheduling for common ai instances in which instances are being stopped at 9PM and started at 10AM.

  • Scheduling is setup from monday to friday only. and on weekend servers will be totally stopped.

  • We have setup lambda function which is triggered by Amazon EventBridge scheduler.

  • Lambda function

    • centralised_ai_instances_schedule
    • In this function, We have created aliases based on env and operation.
    • Now, there are 4 aliases for development and production as mentioned below.
      • For development: scheduleON-dev, scheduleOFF-dev
      • For production: scheduleOFF, scheduleON
  • Created 2 Rules in Amazon event bridge for on and off operation based on lambda function alias. below are the rules

  • Jenkins job:

    • There is 4 jenkins jobs for code deployments on lambda function based on operation like on & off for development & production. Below are the jobs link.
    • Jobs
    • Instances configurations are fetching from s3 in jobs.
    • We store instances configurations like instanceid, instances ontime, offtime on s3 in below location.
      • s3://oriserve-config/Tools/centralised_ai_instance_scheduler/Lambda/Environment/Development/
      • s3://oriserve-config/Tools/centralised_ai_instance_scheduler/Lambda/Environment/Production/
    • Below are the file format
    {
"instanceid":"instanceID",
"ontime": "00:00",
"offtime": "00:00"
}
  • If there is a req for adding/updating servers for scheduling then first download the file from s3 from development folder then add/update your instances details in above format and upload to s3.
  • After tested in development then update the same for production with approval of devops lead.