Dynamic Access Policies for Energy Cost Management of Microservices

Data centers consume increasing amounts of electrical energy, driven by artificial intelligence services, and in many energy markets the price of electrical energy is highly volatile, leading to challenges with cost management. The direct energy costs of computational services offered by data centers are not transparent to their clients, providing neither easy means nor incentives for service clients to reduce operational costs. As a possible contribution to solutions, this paper demonstrates a simple system that can be implemented as an application-level proxy that balances the priority of service requests against real-time energy costs, for example by limiting low-priority traffic during periods of higher prices or by deferring it to periods of lower prices. The principle of operation is similar to rate limiting, and the rate limiting function is precisely governed by formal access policies, consisting of a mathematical function that combines dynamic variables from internal and external sources. To manage energy costs, access policies are designed so that decisions depend on the real-time price of energy, which is obtained from an external source. A proof-of-concept system was developed, and simulations show that even straightforward dynamic access policies can be effective in managing energy costs.