Explore insights on real-time edge computing, AI, and cloud solutions. Learn how our technology optimizes performance across industries.

Blog

First principles thinking is a powerful but under-used approach to building systems to solve complex problems. The basic idea of first principles thinking is to break down complex problems into essential elements that you know are true and then reassemble them from the ground up to solve the problem. In my experience, this approach enables one to unlock creativity and build innovative and robust systems.

Why do folks ignore this? Or maybe folks don’t know how to break problems into first principles and reassemble novel solutions to the problem. I had excellent results with applying first principles thinking to solve complex problems.

So thought I would share my approach in this post to show how one can build systems (and how we built Macrometa) by applying first principles in two different and complex domains — financial markets and 

Building a Trading System from First Principles

Let’s take investing/trading as it is a hot topic this year with inflation and recession worries. The problem is simple - I want to build a robust and profitable trading system that generates profits with less risk & not require too much of my time. Specifically, financial markets (equity, bonds, currencies, commodities, etc.) are complex and vast, so our task is to navigate this vast complexity and solve our problem.

Most retail investors have strategies to trade/invest in the markets. If we distill those strategies, often the first principles unwittingly are:

I’m sure we can do a little better and decompose financial market behavior into three first principles.

First principles for building a trading system

In any time frame (daily, weekly, monthly, etc.),

Each market can move only in three ways: up, down, or sideways

Some markets move opposite to other markets

Some markets move faster than other markets in the same direction.

The above principles are simple and obvious, isn’t it? So, how do we build a profitable trading system from these first principles?

Each financial market can move only in three ways - Up, Down, and Sideways

Let’s take the first principle - Each market can move only in 3 ways: up, down, or sideways. Assuming we trade only on the long side (i.e., no shorting), it means we want to be in the market when it is going up and get out of the market when it is going down.

One way to do this is by using moving / rolling averages. Mathematically, a rolling average of the price will always be below the price, when the price is going up and vice versa when the price is going down.

Buy Rule - Buy equities when the market closes above its moving average.

Sell Rule - Sell equities and go to cash when the market closes below its moving average.

Some markets move opposite to other markets.

Let’s take the 2nd first principle - Some markets move opposite other markets. We know markets go through risk-on and risk-off periods. During risk-off periods, the capital flies from risk investments like equities to safe assets like US Treasuries, Bonds, etc. You can measure this behavior via rolling correlations.

Sell Rule - Sell equities and go to bonds when the market closes below its moving average.

Following are the results of a trading system using the above rules.

Equities Proxy: VFINX (Vanguard S&P 500 Index Fund)

Bonds Proxy: VBMFX (Vanguard Total Bond Index Fund)

Moving Average: 10-month Rolling Average.

Test Duration: 1990 - 2022 (September). Timeframe: Monthly.

One can undoubtedly develop additional first principles to improve this system's results further. For example, using the 3rd principle to compare bonds and cash further reduces risk. You can also try this with other moving average lengths to see the stability of the returns. Finally, some skeptics might question why we’ve used a monthly timeframe and indexes are used in this system. The reason is simple - Inversion Mental Model.

Disclaimer: Anything in this post should NOT be taken as financial advice. Use it at your own risk.

Building Macrometa GDN from First Principles

 is a platform for enterprises to build global systems of interactions and APIs to serve users around the globe instantly. In the past, only a handful of companies, like Google, Facebook, etc., could build these types of systems utilizing an army of PhDs and millions of dollars. Macrometa GDN gives enterprises an easy button to harness their data in real-time and, eventually, contextualize, replicate, monetize and share at a fraction of the cost and expertise.

A few years back, when my co-founder (Chetan) and I set out to build GDN, we faced a challenge: creating an edge native data & compute platform running across the globe in 100s of locations. Further, this platform must run seamlessly across multiple clouds, on-premise data centers, and telecom (5G) locations. None of the existing data or compute platforms could do this.

So our first step was to decompose the challenge into fundamental problems (aka first principles) that we know are true and need to be addressed. Then we solved these real problems and combined the techniques to build the GDN and edge compute you see today.

The first principles that underlie the GDN platform are:

Networks are unreliable across data centers.

Networks have large latencies across data centers.

Concurrent reads-n-writes across data centers.

Query languages are CRUD operations on document fields.

Networks are unreliable across data centers

Unlike networks within a data center, the networks across regions are unreliable. So the first step is to address this and create a reliable data replication mechanism across areas. To solve this, we built a replication system using append-only logs with a pull model in the same fashion as 

A key difference - In GDN, the geo-replicated streams are created and managed in a coordination-free manner. Also, initially, GDN used a SWIM-based gossip protocol over UDP to discover regions, but later on, a hierarchical peer-based discovery protocol replaced it. The primary reason is eliminating UDP to reduce headaches and the platform's footprint.

Have you ever tried to access servers running in the USA from Europe or Asia? If so, I don’t need to explain this principle. The results of a simple HTTP ping test between my server (San Jose, California, USA) and various AWS regions are shown below. The network latencies across regions are in triple digits compared to the single-digit latencies we see within a data center.

The challenge is that the current data platforms are designed to run inside a data center with single-digit millisecond network latencies. These data platforms use synchronized consensus protocols like Zabber, Paxos and Raft, etc., which are chatty and require coordination between the regions. So when you run these platforms across multiple data centers, you will have the following problems -

The operations incur significant latencies to complete because of synchronization between regions.

The operations frequently fail because of unreliable networks between regions. The more regions, the more frequent the failures are.

The operations create significant network traffic because of protocol chattiness.

Macrometa GDN eliminates these problems using a coordination-free architecture, i.e., 

 are used for convergence across data centers. More details on this are in the next section.

This coordination-free architecture allows GDN to scale to many data centers. Also, it enables GDN to serve reads-n-writes with low latency and no coordination with other data centers.

Concurrent reads-n-writes across data centers

Conflict-free Replicated Data Types (CRDTs) are data structures that can be updated without expensive synchronization/consensus. They allow data to be concurrently updated on several replicas, even while offline, and provide a robust way to merge those updates into a consistent state. In this regard, CRDTs differ from the algorithms used by Google Docs, Trello, and Figma, which require all communication between users to flow via a server.

GDN incorporates CRDTs by decomposing a database into its most minor elements (aka 

A JSON document is a collection of data types - string, boolean, number, list, or object.

GDN uses RICE (Real-time Intelligent Convergence Engine) to handle insert, update or delete operations on a JSON document. The RICE engine generates CRDT operations for any changes to a JSON document. Similarly, it applies CRDT operations to the local document state and converges to a consistent state mathematically deterministically. The CRDT operations are replicated across all data centers via persistent streams. Similarly, GDN uses Vector Clocks to tag the CRDT operations for establishing causal relationships between changes across data centers.

A big challenge when using CRDT is garbage collection of the ops history log. The history log of a document grows with each insert, update, or deletion of a field in the document. Some platforms try to do garbage collection by coordinating across data centers. But that is not scalable and introduces the problems described in the previous section. Instead, GDN uses an innovative co-ordination free garbage collection algorithm to keep the document (aka history log) from growing unbounded. You can read more about it under the technology section on the 

Currently, only some systems provide CRDT data structures. Providing a whole database built on the CRDT mechanism is more complex. Unlike individual data structures, to give a CRDT database, the database query language (ex: SQL, C8QL, etc.) needs to be translated in real-time to applicable CRDT operations. This is what GDN does to provide a database abstraction to users while using CRDTs and ops logs internally.

For example, an update SQL query like the one below will be translated in real-time by GDN into CRDT operations shown below:

When a user issues an INSERT, UPDATE or DELETE query (aka WRITE queries), GDN converts these queries internally into a series of CRDT Operations. Then these CRDT operations are geo-replicated asynchronously to applicable data centers for local processing. Each data center applies the CRDT transformations and incrementally updates local materialized views. READ queries are served by the materialized view engine for 

. Various types of indexes (hash, skiplist, TTL, geo-spatial, full text ) are supported on these materialized views to reduce the read latencies further.

Similarly, GDN supports multiple data models (key-value, docs, graphs, search) again by leveraging 

 —> A set of JSON documents with a primary key for each document. Supported operations - get, set, delete

Document Store —> A set of JSON documents with a primary key for each document. Supported operations - Query Language (SQL), Indexing & Transactions.

 —> A set of JSON documents to represent a graph (i.e., each vertex and node of a graph is a document). Node documents contain from and to fields. Supported operations - Query Language, Indexing, Transactions, Graph Functions, etc.

Search Store —> A reverse index on a set of JSON documents. Supported operations - Query Language, Indexing, Search Functions, etc.

Most data platforms are monolithic systems. GDN is built as layers, each building on top of the previous layers. The layer model enables GDN to support multiple query interfaces easily. GDN supports C8QL, SQL, Dynamo, and Redis query languages as plugins. More on the way. So the revised picture looks like the one below.

Today most folks use separate data platforms for data-at-rest and data-in-motion. Example - Databases for data-in-motion and 

 for data-in-motion. The data is the same whether it is data-at-rest or data-in-motion.

So why do developers have to use four different platforms (i.e., DB, Streams, Search, Stream Processing) and waste their time wiring these platforms together, spend days to program and ensure the data is consistent across all platforms, etc.? Similarly, why are architects forced into building complex architectures, and why do operators have to manage & maintain a forest of products?

Like most readers of this post, I was that developer & architect. I paid the tax to deliver my stuff. So naturally, this question was on top of my mind while architecting GDN. Consider a table with one document and a stream with one document. Is there any difference between them? No. Say we have an update to the document. The table contains one document with the latest change, whereas the stream contains two documents. In other words, the table is nothing but compaction of the stream contents to reflect the latest state of a document.

In GDN, every table is also a stream. This opens up influential developers' capabilities and eliminates the need for complex architectures that look like spaghetti balls. A developer can now query the data when it is at rest (aka tables) and the data when it is in motion (aka stream processing). Also, a developer can join tables, join streams or join streams & tables as part of their queries. This is a very nice power to have. Please refer to the 

 section of GDN docs for more information. So our revised GDN architecture looks like the below.

There is a good reason we need to consider this as a 

 in designing data platforms for the future. Data is the new oil, and every nation would like to control it as soon as they can figure out a way to monetize it (or use it to control the population for authoritarian regimes). Also, many citizens (including myself) prefer privacy and control over their data. GDPR, California & India data laws are just a start.

Currently, the data regulation and compliance solution are to set up dedicated clusters in each applicable region and wire them together. It is a very cumbersome and expensive way. Ideally, the data platform should enable developers to do geo-pinning and geo-fencing of data & compute efficiently. Unfortunately, most databases, streaming systems, and stream processing platforms are not designed for this.

GDN is an edge native platform. The bottom-most layer of GDN is about discovering data centers and creating geo-replicated streams between them on the fly. (Please see section - Networks are unreliable across data centers). So we extended this bottom layer to create a streaming mesh overlay between selected regions on the fly, providing containment to all higher layers. We call these GeoFabrics.

 automatically geo-pin the data, streams, stream processing, and compute to only regions attached to that geo-fabric. A user can create GeoFabrics using a subset of regions, all based on their criteria and not pre-generated (such as the US, Europe, or Asia, for example).

There are three advantages of GeoFabrics for GDN users:

Comply with local data regulations (such as 

Reduce read/write costs associated with global replication

Create needed geo-fabrics with a click of a button

The modern web we know today was possible because Akamai, the grandfather of CDNs, pioneered the idea of moving static data to the edge. Likewise, the cloud we know today was possible because AWS pioneered moving data to hyper-scale data centers that host enormous computing power. In both cases, it was data that was moved to compute.

So why do we need to move the compute to data? Four reasons why it is better & easier to move compute to data instead of moving data to compute:

Today, data is an asset that every enterprise wants to protect and monetize.

Also, today many nations want to ensure the data does not leave their borders and require enterprises to comply with regulations and privacy laws.

Similarly, today the amount of data generated is astronomical compared to 10-20 years ago.

Similarly, computing is temporary and available everywhere today.

The industry started taking steps in this direction by providing serverless functions-as-service like AWS Lambda Edge Workers, Akamai Edge Workers, etc. But this is not sufficient for three reasons -

The backend for most enterprise apps is microservices and daemon services.

Most enterprise apps require stateful computing.

Enterprises prefer compute-on-demand and geo-pinning to a subset of locations for compliance & cost-reduction purposes.

Macrometa GDN already has rich orchestration & data management capabilities. So the next step was to enable users to bring compute to data around the globe with a click of a button eliminating the need for complex & costly operations. GDN provides the compute orchestration and colocation of computing & data around the world via its query workers, stream workers, functions as service, and containers as service.

From time to time, people ask us how Macrometa GDN compares to other NoSQL, Streaming, or Stream processing systems. Maybe we have done an excellent job abstracting all the underlying technologies like CRDTs, materialization engine, multi-model, geo-replication, etc. GDN is fundamentally different architecture from current data platforms and is edge native. Most data platforms are not edge-native and cannot scale geographically nor provide capabilities like 

This post provides a good set of first principles and ways of thinking for readers interested in building geo-distributed data platforms. While your design and implementations will vary, the problems needed to be solved are the same to be an edge native platform. Similarly, I hope this post helps readers evaluate geo-distributed data platforms better.

Finally, I hope this post provides good examples of building novel and innovative systems starting from the first principles.

 series, whose theme is, “The future is edge native: Building on the cutting edge with Macrometa and friends.” Join us at Developer Week to read more such exciting posts. And if you want to see Macrometa in action, 

Building Systems & Macrometa GDN From First Principles

Come, get edgy with Macrometa and friends at 

 November 7th-11th for a fun-packed, informative adventure and score some octopus prizes. Get an exclusive first-look at new features, demos, cutting-edge topics, real-life use cases, and a sneak peek into our 2023 roadmap. If you are a developer or architect this is an event you don’t want to miss! Whether you are already “living on the edge,” curious about the edge, or just want the freedom to quickly create and run world-class apps close to your customers, you’ll take away valuable new ideas and insights. 

Join sessions that interest you online or async from your couch or desk. You can even build a streaming app in 10 minutes or less!

See what other developers are building on the 

Learn more about why building and running apps on the edge - close to your customers, partners, suppliers, and staff - can accelerate developer velocity and ensure operational simplicity.

Here are the highlights for each day so you can block your calendar.

Welcome to Developer Week- Monday, Nov. 7

The future is edge native. We will kick off Developer Week talking about the key benefits of application development on the Edge.

9 am PST - Discover how the Edge enables developers to build world-changing apps with Macrometa CEO, Chetan Venkatesh.

12 pm PST - Curious about how Macrometa was built? Macrometa CTO and Chief Architect, Durga Gokina, recounts how first principles were a key part of building Macrometa from the ground up.

3 pm PST - Watch a brief keynote video from Chetan on edge native technology and Macrometa. You won’t want to miss these insights about the Edge!

What’s new with Macrometa - Tuesday, Nov. 8

Be the first to preview details about the 

 that includes all the key ingredients you need to accelerate and simplify application development.

12 pm PST - Dustin Larimer, Macrometa Head of Design, showcases the 

 and shares the process of designing a developer-friendly platform.

3 pm PST - Get more details about SQL (relational and non-relational) support in a demo or blog.

Macrometa and Akamai partnership - Wednesday, Nov. 9

The CDN meets the GDN! Macrometa and Akamai have teamed together to provide an enterprise-ready edge with powerful tools for large-scale apps and APIs.

9 am PST - See how Macrometa and Akamai team together to accelerate app development on the Edge.

12 pm PST - Listen to a fireside chat as experts from Macrometa and Akamai will discuss how data is changing how developers build and what an edge native world means for 2023 and beyond.

3 pm PST - It’s time for a “watch party” with a streaming video app created using Akamai EdgeWorkers and 

Who doesn’t love demos and real-life use cases? Dig deeper into all the technical details from our customers and partners.

9 am PST - Find out what other development teams are building on the Macrometa GDN and even why they chose us over other products.

12 pm PST - Watch Justin Johnson, Macrometa Head of Product, and our friends at 

 as they chat about app development on the Edge. 

3 pm PST - Learn why AppSecEngineer.com chose the Macrometa GDN over Cloudwatch for their developer training platform.

Sneak peek into the future - Friday, Nov. 11

See the future without a crystal ball, we have a lot of exciting things to share with you!

9 am PST - Join our product team as we preview some 2023 roadmap features.

12 pm PST - Hear directly from DISH Network how they optimize real-time workloads with the 5G Edge.

3 pm PST - Tangle shares their experience of why low code is key to quickly adapting to changes and growing faster.

Don’t wait, sign up today for Developer Week

 and receive cool octopus swag and notifications of all the activities. From there you can join what interests you, when it is convenient for you. 

 if you have any questions, or schedule a 

 with our experts if you want to dive a bit deeper. If you are new to Macrometa, try it out today by requesting a 

 to get started. We can’t wait to see you at the Edge! 

Read the press release for more exciting details about Developer Week 2022: 

Macrometa's Inaugural Developer Week Invites Enterprise Developers To Build Apps And APIs On The Edge.

? Come take a look around, leave a review, and say hi.

Calling All Developers To The Edge - Announcing Developer Week

Data is the “new oil” and organizations that are able to quickly analyze both real-time and historical data will have a competitive advantage. But ingesting, managing, storing, and correlating different types of data can become a costly, administrative nightmare.

In this blog, we will walk through details about Macrometa’s Global Data Mesh and its 

, search, and streams components. We will also go over the two other parts of the Macrometa 

 (GDN) - Edge Compute that has features like 

 and Data Protection with features like geo-pinning.

Global Data Mesh: Store and serve any kind of data

, you can use the data model that makes the most sense for your application and make it available locally, globally, or both. The result is a flexible, ultra-low-latency data layer purpose-built for global, real-time, and event-driven use cases.

 is that you don’t have to worry about time-consuming data integrations or storage maintenance. You simply use different databases at the same endpoint - so it is much easier to integrate different data streams into an application. 

In a data mesh model, data can be stored, accessed, and processed where it originates and/or where it is needed. Organizations can use a data mesh to collect and process data where its users are and gather real-time insights.

Data is automatically replicated across the Macrometa GDN, and API requests are routed to the nearest available location. There are over 175 

 around the world. Changes at any location propagate across the network instantly, bringing all nodes to parity within milliseconds. Let’s run through the components of the Global Data Mesh. 

Document Store - NoSQL document collections with flexible indexing and modeling

 makes it easy to read, write, replicate, query, and index JSON data on the Macrometa Global Data Network. Advanced features let you change data feeds for publishing or subscribing to document collections as streams. 

Graphs - Dynamic, high-throughput graph store with ultra low-latency queries

 are a fault tolerant, fully managed graph database service that makes it easy to build and run graph powered apps and 

 on highly connected datasets. You can easily ingest millions of related events across tens of thousands of streams with billions of connected relationships in real-time and query the dynamic graph structure with milliseconds latency.

Streams - Distributed, real-time stream engine with pub/sub and message queues

 integrate seamlessly with your existing workflows, giving you fast, scalable, and reliable access to your data from a wide range of popular frameworks. Build stateful low-latency micro-services and real-time data pipelines. You can create and use a stream on any collection or store. 

Dynamo Store - Feature-complete and fully compatible with AWS DynamoDB

 offers a low latency, globally replicated key-value and document database that is serverless, multi-cloud, multi-region, and multi-master. Macrometa Dynamo mode offers all the features of AWS DynamoDB with a 100% compatible API and wire protocol proxy (you can even use the AWS SDK with Macrometa Dynamo mode). 

Key-Value Store - High-throughput KV collections

 lets you build high-performance web and mobile apps and APIs quickly and easily without the need for managing anything. Use our simple API to quickly add state to your AWS Lambda functions, Cloudflare workers, or container apps. 

Search - Flexible full-text search and similarity ranking across multiple data stores

Search has become such a critical part of application development and is key to improving customer experiences. 

 offers powerful full-text search and real-time ranked search with query parameters like spanning phrases, wild cards, images, proximity, and similarity. Full-text search is available in 20+ languages with dedicated analyzers and advanced features like stop word removal and stemming.

Bringing it together with Edge Compute and Data Protection

What separates Macrometa from other data storage solutions is that it also brings compute closer to people and where data is generated, regardless of where that is. Macrometa’s Edge Compute puts data into action and delivers real-time and event-driven apps and data in <10ms to 80% of the global population. 

 lets developers create simple REST APIs on top of the data mesh. 

 enables Complex Event Processing (CEP) functions and 

 workloads - in minutes instead of days or months. 

Stream Workers play an important role with data collections and stores. With Stream Workers, data could come from different sources (GDN streams, GDN Database, HTTP endpoints, Kafka, and MQTT etc.) and can be published to different sinks or stores. 

 also allows you to transform data from one data type to another, aggregate streams, and clean and filter data. This CEP engine automatically creates objects like streams, collections, and even tables and remote databases. 

Macrometa’s Data Protection supports geo-pinning or geo-fencing to address data sovereignty and privacy use cases. Developers can set up 

 for regional and global data, and then adjust locations as needed. Macrometa offers three different types of 

 (user, token-based, and API keys) and is 

With many tutorials, Quickstarts, and easy-to-follow example 

, you can get started with Macrometa’s Global Data Mesh in minutes. 

Say Yes To The Data Mesh: Explore Multi-Model Data Collections And Stores At The Edge

Cloud is mainstream today, and its evolution ushered in several new businesses, business models & jobs like SRE, DevOps, CloudOps, etc. But conceptually, the model is a client-server model of the 1980s with increasing distance between client & server in each decade, i.e., initially, it was Client → Server (where servers were a few feet from the user), then Client → Data Centers (where servers were 10s and 100s of miles from the user) and finally Client → Cloud (where servers were 1000s of miles from the user).

On the other hand, the demands of enterprise businesses today are very different from what they were in the 80s and 90s. In the 80s, software was a niche area. Today, it is the main driver for companies, and survival/failure depends on how well the company can see the shifts, utilize the right tech and innovate.

In the '90s, 100 MB of data is a big deal. I still recall the excitement of getting 5.25-inch high-density floppy disks that could store 1.2 MB of data. Today, 100 TB of data is frivolity.

Similarly, in the '90s, the concept of digital data privacy or data regulations was insignificant. Today, we have the GDPR law, California Law, and many more regulations to secure data.

Also, can you recall how many enterprises in the 80s and 90s were providing services globally, dealing with millions of users, not many, right? Today, pretty much all companies have their business serving global users.

Companies that could see the shifts early in the 2000s and 2010s adopted the cloud when it was in infancy and benefited handsomely (ex: Netflix). While others found seemingly rational reasons not to adopt and got decimated (ex: Blockbuster). In my opinion, enterprises today have a choice - see the seismic shifts happening now and embrace the next evolution or get left behind due to fear.

This blog post concentrates on how these shifts translate to what is needed beyond the current 

From client→server days in the 80s to current cloud computing, the default approach is to move the data from the source to where the services are running. Whether the distance to move the data is 100 miles or 10,000 miles, it does not really matter.

The challenge is that new-age enterprises generate data in terabytes, petabytes, and zettabytes. It’s not just IoT / 

 / Smart Manufacturing, but it also includes retail, gaming, and other industries where millions of users request or generate data via forms, clickstreams, logs, etc. Moving this data 1000s of miles to a distant cloud location is a recipe for a bad user experience, a slower time to act, and a host of other issues like low reliability, network congestion, high computing costs, 

 issues, etc. It does not matter whether it is event-driven or request-response workload.

A better alternative is moving computing closer to the source of the data, especially for use cases that deal with interaction, personalization, 

, and real-time log processing, etc. This will help process the data closer to the source and turn data into dollars quickly by providing a better user experience, personalization, faster actuation, and reduced egress and other cloud costs. But, there are use cases where the moving data to compute makes more sense, and the current cloud model is a good fit. Example - 

Currently, enormous regulatory activities are happening around the globe to protect their citizens (or to monitor their citizens if it is not a democracy) from cyber crime and attacks. It is safe to assume that we’re just starting on these regulations and compliance path and it’ll only become more taxing in the coming years.

On the other hand, today's data platforms don’t deal with privacy. Similarly, most enterprise services approach privacy as an add-on at the end to comply with regulations. The challenges that come with doing it in this way (as an add-on) are poor architecture, humongous development & operational costs, technical complexities, and many more.

So, even in this scenario, a better alternative for data platforms (databases, search, file systems) is to natively support privacy capabilities so that architects and developers can leverage them right from the beginning. This can enable developers to easily geo-fence the data and compute to designated locations, intelligent routing of requests, real-time tokenization, pseudo-anonymization on demand, etc.

Best way to understand this use case is with an example.

Say we’re at a coffee shop and want to grab some coffee, you go ahead and order like - “One Espresso Macchiato in a tall cup, please”. I order with a straight face and order like this - “1 cup 2% milk, one shot of espresso, one tablespoon of sugar, one dollop of foam mixed with these and heated for 3 min, then pour in a tall cup, please.”

Interestingly the latter approach is what most people do when building their services to achieve business outcomes. The following is a sample reference architecture to build an e-commerce application using AWS services -

e-commerce application architecture on AWS cloud

Building apps using point services like the above is akin to how I ordered coffee when we went to a coffee shop. Unfortunately, the cloud providers have done a great job convincing many folks that this is the right technical approach. The folks who benefit from this model are cloud providers, consultants selling expertise, and point solution vendors. It may not be for someone who’s a CTO/CIO/VP/Architect, who needs to deliver outcomes with the least complexity and higher developer velocity.

Again, a better alternative here is to utilize converged services that provide multiple capabilities (like database, search, graphs, streams, and 

) from one interface on a single copy of data. Cloud providers cannot do this from a single copy of data because they have to re-architect their services.

Today, for a developer to run a service, the first thing they need to do is select a cloud and within that cloud a specific region to deploy and run the service. There are two big reasons for this -

Cloud services are proprietary and hence making it very hard to do multi-cloud.

Archive

Building Systems & Macrometa GDN From First Principles

Calling All Developers To The Edge - Announcing Developer Week

Say Yes To The Data Mesh: Explore Multi-Model Data Collections And Stores At The Edge

Beyond Cloud Computing: The Advent Of GDN

How To Scale To “Trillion Events Per Minute” - With Ease And Without Downtime

Macrometa At Current 2022 Kafka Summit Organized By Confluent

Tutorial: Implement Complex Event Processing In Macrometa Within Minutes

Reinventing The Wheel: How DevCycle Made Saving User Data Fast With Macrometa

Choosing A Complex Event Processing Engine: Macrometa Vs Apache Spark And Apache Flink