Explore insights on real-time edge computing, AI, and cloud solutions. Learn how our technology optimizes performance across industries.

Blog

The last 15 years have seen several successive waves of big data platforms and companies offering new database and analytics capabilities. Leveraging the public cloud, these large scale distributed data systems work well in centralized, single region or single data center topologies and are geared for 

Companies such as Cloudera, Snowflake, and more recently Databricks are examples of successful companies that have innovated with novel “read architectures” that reduce the structural cost of reading immutable data and built new big data orientated analytics apps and capabilities by exploiting their lower costs of reads.

We are now faced with new challenges with workloads and use cases that are 

. The write path is far more challenging than the read path. This is because the read path is built on immutable or non-changing data. The write path necessarily involves mutable data with varying rates of change and growth in the data sets of use cases.

Additionally, the write path today has also evolved from the client/server’s “request – response”based interaction model to new, streaming data architectures where streams of “events” need to be processed as they are created or generated by various sources.

A similar cascade of innovations is needed on the write path to solve the structural costs of data writes as big data platforms have done to change the structural costs of data reads.

We summarize the challenges of the write path in modern cloud data platforms as follows:

Expensive Writes - (Ex: DynamoDB - writes are 16x to 64x more expensive than reads).

Centralized architectures - Introduces large access latencies for globally distributed users and as well as large transit latencies and network transfer costs for data to be shipped to the cloud.

Require network communication intensive and coordinated approaches to consistency such as using state machine replication or consensus and therefore are only feasible within a single cloud region (due to the need for ultra-low latency and reliable data center class networks).

Require synchronous replication to ensure all nodes in a distributed system make forward progress on writes with transaction semantics where either all related writes are accepted Macrometa 2or all are rejected. In combination with point #3 above, replication and coordination end up being the Achilles heel for write-intensive distributed systems.

Today’s enterprise apps and cloud workloads are no longer “read intensive”, but are a combination of:

 (ex: IoT, Monitoring, ClickStreams, Fraud Detection etc) or

This is because modern cloud native architectures don’t use just one database behind them for persistence but instead use several, specialized databases and data stores for different types of semi structured and unstructured data. This “polyglot persistence” pattern used by modern apps further exacerbates the write path and write intensity problem as now a single application level write generates a successive cascade of amplified writes to multiple data stores underneath the application.

 pattern where an application level write results in several cascading updates to multiple independent databases or data stores causes very high levels of write amplification and resulting costs.

Attempts to solve these problems using conventional techniques of “write back” and “write through” caching no longer work for such polyglot backend patterns – one simply cannot buffer writes in a single buffer and fan out to multiple databases and data stores underneath without completely giving up on consistency and transaction semantics.

For use cases where multi-region data, or edge based data processing are needed, the problems are further exacerbated by the physical link latency, topology and reliability of the wide area network. These use cases have to contend with well known problems such as:

Given the challenges just described, current centralized distributed data systems and the technologies that underpin them cannot be generalized well to fit edge & multi-cloud workloads.

Macrometa has been now for several years, focused on solving the challenges of the write path.We don’t just want to solve the cost and consistency problem of the write path but also enable globally distributed stateful apps & web services that can run in 10s and 100s of regions world wide concurrently with less than 50 ms end to end latency for data access by 95% of the world’s population.

This is a non trivial computer science problem because:

It requires our platform to run in wide-area deployments where nodes forming the database may be separated by more than 100ms of latency and unreliability in the network connections.

It requires our platform to run across 10s to 100s of data centers and yet present a single system image (SSI).

It requires us to have a significantly better write cost structure and cost-performance ratio than current cloud data systems built on centralized architectures.

Macrometa accomplishes this with a novel architecture that combines geo-distributed, coordination free write and replication techniques and a multi-modal data platform with tunable consistency levels to solve these problems in an edge-native way.

Solving The Trillion Dollar Data Problem In Edge Computing

Today I’m incredibly pleased to share that 

 has invested in Macrometa. Many of you already know of Pelion as a hugely successful, storied venture firm based out of Salt Lake City, UT. Pelion has previously backed Internet blockbusters like 

 and Riverbed from startups to public companies through frenzied hyper-growth and market making. Macrometa is delighted to team up with 

 on our mission to bring distributed cloud infrastructure and edge computing to developers everywhere.

I’m a big fan of humble investors who walk softly but carry a big stick (the stick here refers to an 

 on the startups they back). Pelion exemplifies this approach. From the many meetings and discussions with Blake, Chris, Ben and Zain– it was clear to me that Pelion was a team we wanted to work with. Their powerful curiosity for the end state of edge computing that Macrometa enables, coupled with first-hand experience of having built several category defining businesses like 

 (acquired by Okta), Riverbed and others, lead to a collaboration and new investment in Macrometa, despite us having closed our seed round of 

Incredibly smart, accomplished people who invest early to help founders build category defining companies.

Nowhere is this more apparent than in Pelion’s backing of CloudFlare as one of its earliest investors. Pelion helped finance Cloudflare’s monumental rise into a behemoth in security, CDN and now edge computing. They gave the founders of Cloudflare great latitude to pursue their vision and helped with their advice, network and experience where needed to create results that are self-evident. CloudFlare is today an incredible business with a 50% CAGR on revenues and a 

Passionate about solving complex infrastructural issues buried deep inside the Internet

Pelion backs teams that are going after hard problems. Take Riverbed for example – Riverbed arrived on the scene when WAN latencies made Remote Office / Branch Office connectivity an expensive and unusable nightmare. Riverbed transformed the enterprise’s ability to use the Internet to connect people, applications and data with its transformative WAN optimization technology. Riverbed’s approach was radical for its time and few investors understood what the implications were. Pelion saw that Riverbed’s proposition was transformative and powerful, and helped Riverbed quickly grow into a networking infrastructure powerhouse.

Humble, hardworking and heartful investors

Having worked on multiple startups before, both my co-founder and I decided early in the Macrometa journey that we only wanted to work with people who shared our values for humility, hard work and heartful engagement. We have been incredibly fortunate to get the best investors involved with our business starting with 

 leading our pre-seed in 2017, and most recently 

 leading our oversubscribed Seed round (with support from Partech, Fusion Fund, Sway and others in our investor syndicate). We now look forward to Pelion joining our journey and mission to open up the edge cloud to powerful new applications built by passionate developers everywhere in the world.

If you're interested in a role at Macrometa, we have 

 in our R&D, Site Reliability & Engineering (SRE), Sales (inside and outside).

Announcing Pelion’s Investment In Macrometa

Rob Fry is CTO of Armorblox and board member of Macrometa. Rob has a storied career in Silicon Valley having been at several noteworthy startups like Jask (acquired by Sumo Logic) and prior to that one of the architects of Netflix at a critical time in its life as it transitioned from a DVD by mail business to the global streaming giant that we know of today. Every time you netflix and chill - you can thank Rob for the experience.

While all the research says companies are accelerating migration to public cloud (

), what’s lost in the noise is how companies are inefficient at building infrastructure once they get there. 

 has validated the customer’s pain. The study reveals the frustrating customer story as more and more loads are moved into public cloud. Highlighted in the study is how customers need help to deal with cost issues, optimization needs for cost savings, and expertise and technology to deal with complexity. To add to customer angst, as companies desire to move more loads into public cloud, they additionally want to do so with more than one cloud service provider to commoditize and diversify the services but increases the challenges mentioned above.

One of the paralyzing realities in customers' daily lives is an inability to hire people with cloud expertise to help solve these issues. Even though public cloud has been around for over a decade, this extended period of time has not translated to providing a sufficient supply of experienced people to meet the business need for proper public cloud adoption.

In the years since I left Netflix, I’ve seen the customer struggles first hand and experienced them myself at companies I’ve led. Working at Netflix was an opportunity that showed me how to leverage public cloud 

Netflix adopting public cloud was an evolutionary step as a primary business model for a company. Similarly, today’s evolution in public cloud technology can help be a driving force for companies to be successful in building cloud-based architectures. Typically taking the form of a Platform-as-a-Service (Paas), these solutions abstract away complexity to provide optimized technical architectures that span globally across different public cloud providers while reducing costs and filling in the void of missing technical expertise.

Earlier this year, I was introduced to one such team & technology. At first glance, their technology seemed to be an improvement and more easily implementable approach to what is commonly referred to as Edge Computing. As I spent time with the co-founders and the technology, I realized the noticeable separation of their offering compared to the competition. If you’ve spent any time studying the Edge Computing space, you’ll quickly come to realize this term has different definitions depending on the use case and customer. However, this company had technology that could solve all of them with an impressive and broad set of exceptional customers to validate them.

With the excitement of realizing the potential for both the team and the technology, I’m delighted today to be part of the public announcement of the DNX Ventures lead in the seed+ round investment in Macrometa for the next stage of growth for this successful edge computing platform.

It has been such a pleasure working with the Macrometa team. Co-founders Chetan Venkatesh (CEO) and Durga Gokina (CTO) are the right mixes of intelligence, creative passion, understanding of strengths and weaknesses in the space, with high-achieving results. Watching their impressive traction and the vision of what this company could accomplish in the space will be fun to watch.

Now is the right time for the Macrometa team and their technology. Public cloud growth is increasing. Businesses desire to handle the challenges in front of them better and require optimization and reduction of costs. All will drive adoption in the edge computing space. The expertise Macrometa brings to supplement existing engineering initiatives along with Macrometa’s technologies capability of simplifying architectures for faster innovation at a lower cost will be a few of their keys to success.

And in the future, I think we will look back at the year 2020 as the beginning of a transformative change in what it means to build and efficiently leverage the public cloud.

I am excited to join this team as a board member and enjoy contributing to Macrometa’s journey to make edge computing ubiquitous.

Rob Fry On Why You're Doing Cloud Wrong

 is the former CEO of Identity Authentication Management Unicorn and disruptor 

 - an edge centric AI tools company acquired by Apple. Jon serves as independent director on 

I like products that make its users heroes. 

 does just that, and it makes me proud to be a new BoD member there.

Macrometa’s GDN(Global Database Network) is, among other things, a CDN (Content Delivery Network) that can deliver consistent live data globally with milliseconds latency anywhere in the world out of a Read-Write distributed database. So a write in Vietnam can be read by an app in California in a few mS. And there’s more: Durga Gokina and his engineering team realized that a table can be a stream, and made it so that it enables high-performance and inexpensive global stream processing in order to ingest, process and augment or transform data at the edge in real time. The app in California gets updated in mS with a push if it wants with no read required. In fact, with the GDN, queries can now become serverless API calls deployed globally.

The GDN at its most basic is a really great CDN attached to a globally-consistent distributed database, but when fully exploited is a super-fast, fully-featured, compute-at-the-edge backend.

And it gets better: Macrometa’s GDN is an order of magnitude cheaper than today’s distributed databases like DynamoDB or MongoDB, and it’s a drop-in cost-saving cache for them. Who doesn’t like cutting their AWS costs by 80%+ with a few hours of effort?

Enterprise SaaS startups win when they have a service that delivers more capabilities that can be deployed in an MVP in hours, and is significantly cheaper to run than incumbents, coupled with a high-performance team to build the product and the company. For me, all of these boxes were checked at Macrometa, so it was gratifying to have the opportunity to join its BoD. Macrometa’s GDN marks an inflection point for how scalable apps will be developed and I am proud to be part of the journey.

Jon Gelsey on "Why I Joined Macrometa's Board of Directors"

, a writer and artist from Cleveland wrote something that I've pondered often in the last few months. She asks, "Do you think the universe fights for souls to be together? Some things are too strange and strong to be coincidences."

In the world of business, discussing souls, togetherness, and coincidences is taboo. Yet as I sit here writing these words - I'm struck by the profound serendipity of just having raised a 

 and all the wonderful lessons, people, and experiences that have happened in the short time since closing the round.

 by our first investor, board member, my extraordinary and dear friend, 

. Yash led our pre-seed financing in 2017 and took a risk on a crazy napkin sketch of a database that would run in 100s of locations worldwide. I've been pitching products, companies, and ideas for almost two decades now and know the fundraising game at some depth. I've met hundreds of funds, principals, and partners over the last 15 years. Yet here I was, meeting Q over zoom and unable to put my finger on the incredibly rich presence, thoughtfulness, and authenticity he possessed. A few days later, Q invited me to join him on a hike (socially distanced) to talk more. What unfolded on that hot June day by the San Francisco bay's foaming sands in Foster city was incredibly special. As Q and I spoke - it became very evident to me that he and DNX are special. The valley and Sand Hill Road are full of brilliant, accomplished, and ambitious people. But Q was more than brilliant, accomplished, and ambitious - there was a "Je ne sais quoi" about him - I still don't know what it is, but I've put it down to a powerful combination of authenticity, a genuine curiosity in people and a very refreshing self-effacing honesty and humility that frankly far too few people on Sandhill Road possess.

Today, I'm incredibly pleased and excited to announce our oversubscribed $7M seed financing lead by DNX. All our incredible investors from the last pre-seed round participated - my heartfelt gratitude to 

 (Nitin) for underwriting the crazy napkin sketch in 2017.

 are joining our board of directors as independents. Rob was one of the architects of Netflix as it transformed from a DVD by mail company to the global streaming giant and viscerally faced all the problems we strive to solve at Macrometa while building Netflix. Jon was the CEO of Auth0 and Xnor.ai (acquired by Apple recently), and he invented and wrote the book on building products and companies that primarily sell to developers and architects. At Macrometa, we do just two things: (1) We build great products, and (2) We sell great products. We are incredibly fortunate to get Rob and Jon's help to build and sell products we hope will enable developers to do things locked in their imaginations and held back by the complexities and arcane architectures of the cloud.

Finally, I'm super excited to say Welcome to 

, who joins us to lead our marketing, developer experience, and relations. Justin is an inspiration and a hero to me - I've followed his work at Keen.io and then at StackPath, where he helped build and market exceptional products and then formed vibrant developer communities around them.

Durga and I would be remiss if we didn't thank our team members for their hard work from the first day we started in 2017. To the Macrometa family spread across the US, Europe, and India - congrats to all of us - the edge is our oyster.

 in engineering, marketing, sales, and customer success - we don't care where you live or work from - we have always been, and will always be about enabling you to do your best work from where ever you are happiest doing it).

Announcing our $7M Seed raise lead by DNX

", Josh Breilinger, a VC at Jackson Square ventures, makes a powerful argument that one of the most effective forms of competition comes in the form of newcos who aspire to take chunks out of an existing platform by better addressing the needs of a specific vertical within that platform

In the past, I've viewed edge computing as a new kind of tier to cloud computing and one that potentially eclipses the centralized clouds - because it's a natural place to run anything that is interactive, needs to analyze large volumes of shared state data and quickly compute against short time value data. Seen from Josh's perspective it almost seems inevitable to me that edge computing will eclipse cloud computing and that new companies will not just challenge but dethrone the big cloud incumbents.

The incumbent cloud vendors will always see edge computing as a specialized sub market to cater to - with extensions to their current technology stack. And there lies the fallacy, that the current centralized stack can be extended to work with edge computing. Edge computing demands its own architecture - one that eschews centralization for distribution, with the goal of creating revenue instead of saving money.

Event driven programming is wasted on centralized clouds. It is at the edge that event sourcing and reactive technologies really open up trillions of dollars of possibilities in real time, low latency edge computing - in robotics, data analytics, gaming, smart things and machine to machine interactions. When events can be detected, processed and reacted to in single digit milliseconds, the cloud of today will seem quaint and fill us with nostalgia for a simpler time much as a Casio LCD calculator does when compared to an iPhone. 

Just as AirBnB took a submarket of Craigslist and turned it into a standalone multi-billion dollar a year business, Macrometa and other edge computing startups will take today's submarket of edge and turn it into its own hundreds of billions of dollars a year of business in the future.

The "Edge" Is A New Kind Of Vertical Cloud 

Ages ago I had the privilege of building out the first field engineering team for VMware’s Cloud Service Providers. Back then it was an easy message for users to understand – we could take your existing workloads and give you improved resilience and operational costs, while providing the flexibility to deliver enterprise applications in a familiar way.

Since then, a few things have changed in our industry.

User communities have grown, mobile and distributed applications have become a real value to Enterprise. The current iteration of that transition is edge computing, which not only distributes the app closer to user populations enabling things like data privacy, but also drastically improves performance. The days of centralization for resilience are over.

What about operational costs? When the options were managing it yourself, sourcing talent, and buying infrastructure, 

 made a ton of sense. We’re past that now. Distributing applications to the network edge doesn’t change that dynamic, but it does create operational efficiencies for your users which ultimately increases productivity and either revenue or the marketability of the application.

This is the basic thesis for Content Delivery Networks (CDNs). You take the data that can be placed closer to the user and just do it. No more long transit times and transit costs to fetch an image for a web page or a key from a Key/Value store. Put it closer to the user and enjoy the benefits of increased performance and efficiency. We needed a similar paradigm for dynamic data, and that is what Macrometa is all about.

The cloud model holds true for Macrometa. We can now provide even better resilience, reduced operational costs, and still deliver applications in a familiar way – while allowing developers to build apps in a familiar way. From day one Macrometa has been committed to supporting the stacks and frameworks that developers are comfortable with. To put it simply, build your app in the same way but deliver it faster and with more end user performance by leveraging a ready-made backend that is edge-native.

The cloud will continue to be relevant for a long time due to its ability to quickly stand up infrastructure and scale it. But the new reality is that the way we access data is becoming just as important as the way we host it, and for some applications like CRM and mobile apps it is even more important to reduce the costs and delays that backhauling to a cloud datacenter creates. Basically, we're reclaiming the value of resilience and operational costs that cloud gave us 10 years ago and applying it to the brave new world of edge applications. 

Are you ready to take the next step and join our community of hundreds of developers building the next generation of apps? Click on the link below to create a free developer account on Macrometa’s Global Data Network.

The Cloud Is Now The Limitation

What if every food option you ever wanted was just a few miles away?

Spending my career in Dallas, TX while working for companies in California has been fun and offered me the experience of traveling all over the world. Business travel is anything but glamorous – if you’ve done much traveling you’ve probably shared my experience of watching the last flight out of a random city be cancelled due to weather somewhere else on the continent while you’re desperately trying to get home before a holiday weekend.

Having said that, there are some perks. One of these was In-N-Out Burger. For decades this franchise was a California phenomenon. It was another place like Shake Shack, NY Pizza and White Castle. Delicious and totally unavailable in Dallas. I enjoyed the opportunity occasionally to grab a great In-N-Out Burger in California, but that highly centralized model meant if I was sitting at my home in Texas and wanted one I was completely out of luck. It was the opposite of fast food for me, and required a lot of planning and waiting to make it happen.

On a brighter note, over the last few years they’ve been popping up in Texas. Now there are 3 convenient locations near me, so I can grab one any time I feel like it (or more appropriately, my kids can have one now instead of listening to me tell them it is really good… if they only had the time and opportunity to make the trip out West.)

Hyper-locality of resources is a huge consumer benefit, and a way to service and acquire customers in a much shorter amount of time. When it comes to applications and backend services, there really isn’t much difference between my experience with In-N-Out Burger and my experience with companies like AWS and Google. It’s a great service, but requires me to cross huge distances to access what I need. To put it another way, when I’m hungry in Dallas the In-N-Out in Santa Clara doesn’t do me any good, and even if I decided to go there I’m going to be starving by the time my flight lands. If I need data in 10-25 ms, then traversing a thousand miles also doesn’t make sense.

When my apps are hungry for data and performance, a datacenter a thousand miles away isn’t satisfying. What if instead of using 5 or 6 fixed datacenters in distant locations (that I didn’t even get to choose) I could use 5 or 6 datacenters right here in Texas? Now my app is blazingly fast.

That’s the value of Macrometa. Our globally distributed edge network provides 5 or 6 datacenters 

 instead of forcing me to perform a costly data backhaul and wait for functions to cold start somewhere in a faraway cloud. The end result is a massive improvement in application latency and user experience.

Macrometa has been proven to eliminate over 240-425ms of backend round trip latency for apps that need dynamic content at the edge. For developers who know where their user populations live across the globe, they also have the freedom of selecting 

 datacenters serve their app. Why should I shoehorn my app into a geographically limited design, when I can run it 

? This is hyper-locality, and it is the foundation of a concept Macrometa has pioneered called the Global Data Network (GDN). Like a CDN for static data and global footprint, the GDN is purpose built for dynamic data 

 global distribution - unlike existing cloud datacenters which are built for local scale and centralization.

The GDN provides multi-model database capabilities including Doc, Graph and Time Series, along with global pub/sub, Streams, and rich APIs. End users never have to send data requests a thousand miles away and wait, they can make local requests and get local performance from the GDN.

Want to see what Macrometa’s GDN can do for your application? Sign up for a free developer account by clicking the button below.

The Merits Of Application Hyper-Locality

A friend of mine was a construction contractor and his customer mantra was simple:

“I can build it good, fast, and cheap. Pick 2.”

You’ve probably heard that before, and of course it applies whether you’re building a house or a cloud native application. The hard part is picking which two criteria are most important for you, and knowing which criteria is expendable. For my contractor friend, if the customer picked fast and cheap his mantra to the construction crew became “it doesn’t look perfect, but it will look fine from the next job site.”

For developers, there is no shortage of tools and SaaS/PaaS products to help them build. I started thinking about how difficult it must be to sort through the available options and select a product that you will essentially be married to for months or years, and the difficulty of balancing quality, performance, and cost.

To get some perspective, I sat down with our Chief Architect to pick his brain about the vendor selection process. In the enterprise space, many product decisions regarding commodity technology are about “good and cheap” – or at least good enough. To put it another way it is about comparing value to price and finding an acceptable compromise. In the developer space, there are other nuances.

One of the big takeaways from the conversation is that cost is measured in different terms by developer organizations. We typically think of cost in terms of licensing, supporting infrastructure, or monthly SaaS charges. As Durga pointed out, it isn’t quite so straightforward for a dev shop.

“Obviously the first selection criteria is ‘can it handle my workload.’ Sometimes the workload itself narrows the field down to just one or two products that can accomplish the task. An example requirement we’re familiar with would be a database that is consistent across a dozen global regions. A hosted NoSQL or cloud platform that can only replicate across 6 locations may be more common, but if I truly needed more locations or a global footprint, then Macrometa would be the right platform.”

This is where the conversation became even more interesting. I asked if there were multiple products that fit the workload needs, how would he narrow down the options and make a final selection?

“The most important thing is the cost – but not in terms of the actual product pricing. Unless the cost is vastly different, the monthly or one-time expense isn’t enough to influence a decision. I evaluate products based upon my most important costs – Time.”

This made a lot of sense to me. If Product A is $500 and Product B is $700, but Product B will let me produce my product (an app) 50% faster, then the additional time to market savings greatly offset the additional expense. Tying up 10 developers for 2 months is better than committing 10 developers for 4 months – not to mention the loss of use or revenue impact of unnecessarily delaying an application release.

“The second most important element is simplicity and ease of use. This is also a major contributor to the true cost in time, since it is faster to leverage existing expertise on the team without having to learn new skills. The same is true for customer support. If we run into issues, it is imperative that the solution be well-documented and that help is readily available.”

Macrometa shines in the time to value category. Providing a ready made, geo-distributed, global data network (GDN) that can serve as a cloud backend across 100s of locations means that a developer's time can now be 100% focused on getting code completed and products shipped. If you’re interested in seeing just how easy it is to accelerate your development cycle with Macrometa and your team’s existing coding skills, just click the button below and create a free developer account.

Good, Fast Or Cheap - Pick Two

Macrometa is the hyper distributed cloud platform for enterprises building global applications at the Edge. 

PhotonIQ

Global Data Network

Learning Hub

Case Studies

Developer Hub

Demos

Support

About

Partners

Press

Careers

Contact Us

Terms of Service

Privacy Policy

Trust Center

Announcing PhotonIQ: The New AI CDN For Accelerating Apps, APIs, Websites and Services

Archive