Explore insights on real-time edge computing, AI, and cloud solutions. Learn how our technology optimizes performance across industries.

Blog

We are living in a world where the way people do business gets transformed into digital form. This phenomenon called digital transformation has resulted in propelling enormous amounts of data into digital repositories at unprecedented rates. The data can be of various shapes, sizes, qualities, and values. Indexing and searching capabilities can contribute to an organization’s success by allowing them to quickly and accurately locate specific information from large data repositories.

Search is often an underutilized feature by application developers. Typically developers use the term "search" to refer to the scenario of searching for a string. But the advanced search capabilities within the 

 enables sophisticated search. One of the goals of this blog series is to help application developers to utilize the full set of benefits provided by GDN Search. 

The first blog in this three-part series about seven different search patterns frequently found in the Macrometa GDN, we will introduce Macrometa 

 capabilities and go over exact value matching. The second blog will cover details about prefix matching, full-text token search, and phrase and proximity search. Lastly, the series will conclude with range queries, faceted search, and geospatial search. Let’s begin by providing a detailed classification and then walk through the first example.

The Macrometa GDN is a multi-model, streaming 

 data processing capabilities. When it comes to search, GDN brings two main advantages over other data stores.

First, the GDN is equipped with a state-of-the-art indexing and 

 facility that enables users to perform sophisticated search operations on 

 pairs, documents, and graphs. Users are not required to reformat the data and upload them to a separate system. Instead, the persisted data on GDN can be directly used for search as well. The complexity of the search can vary — from simple search queries such as exact value matching and range queries — up to complex search operations such as faceted search, and geospatial search. Each of these different categories can be classified as a search pattern.

Secondly, with the Macrometa GDN the search indexes are updated globally whenever their underlying data gets updated. This allows the GDN to answer search queries within a very short period of time with high accuracy.

Exploring the dataset sample for search patterns

The example in this blog uses the London-based hotel reviews 

. The refined dataset has 10,000 reviews collected by crawling a leading travel portal. Each of the data items in the dataset follows a schema shown in Figure 1.

. After downloading the JSON file, replace <DATA> in Listing 1 with the content of the JSON file in the following command. The dataset can be imported to your GDN federation issuing the CURL command on a terminal as shown in the Listing 1. Before executing the following CURL command you need to first create a 

 named Hotels in your GDN federation and then create a document collection called hotel_reviews within that fabric.

Listing 2: Importing a single data item into the GDN federation via invoking the REST API of a GDN node

In the above example in Listing 2 we have imported only a single review made for the hotel named The Savoy by specifying its JSON content. The values <HOST> and <BEARER_TOKEN> refers to the host name of the GDN node and the bearer token can be copied by referring to the REST API of the GDN node.

The overall architecture of GDN Search can be summarized as follows (See Figure 2). The search is based on the concept of views which can be regarded as virtual collections. A view consists of an inverted index which provides fast full-text searching over one or multiple linked collections. The view also stores the configuration information for implementing the search capabilities such as the attributes to index. The view can use multiple, if not all attributes of the documents in the linked collections. Views can be created and maintained via the Web user interface, HTTP REST API, or via the Javascript API of GDN. All the examples listed in this blog series are based on the former two techniques.

Views can be queried using C8QL via its SEARCH operation. Search expressions are composed of the fields to search, the search terms, comparison and logical operators, as well as other GDN Search functions.

Figure 2: Overall architecture of GDN Search

The input values are processed by constructs called Analyzers. These can normalize strings and tokenize text into words, which enables various possibilities to search for values later on.

The search output can be sorted either by their similarity ranking to return the best matches, using popular scoring algorithms (TF-IDF, Okapi BM25), or by using the user-defined relevance boosting and dynamic score calculation.

In the most basic version of a search pattern you could try to match the presence of an exact value. The exact value can be either strings, numbers, number ranges, or booleans. Here we can index and search strings using an identity analyzer. The view used for exact value matching can be defined as shown in Listing 3.

It should be noted that since the default Analyzer is the identity it is not necessary to set the Analyzer context with the ANALYZER() function here. Therefore, exactly the same results can be obtained by executing the query shown in Listing 5.

Exact value matching can be conducted considering several item values. There are three approaches to do this — either using the logical OR operator, using the IN operator, and using the bind parameters.

Now that you are familiar with the Macrometa GDN search architecture, you can get started on exact value matching based on this example. View the 

 in our search patterns series to learn about prefix matching, full-text token search, and phrase and proximity search.

¹The GUI displays up to 1000 records. Complete results can be found via Macrometa GDN's REST API.

Expand Your Search Capabilities With The Macrometa Global Data Network  

While companies today want to collaborate throughout their organization and work closely with business partners, they might need to limit access to documents in a collection to meet 

, privacy, and quality control requirements. 

 is a fundamental component to help protect confidential information and minimize risk to your business or organization. In this blog, we go over the details of the User Access Control feature and how to set up who can view or modify data at the document-level and collection-level.

There are multiple ways admins can apply User Access Control attributes to their 

. These attributes can ‌limit access to documents at run time in 

Govern access by both users and API keys.

Document-level and collection-level access control with attributes

Let's use an example to illustrate how you can use attributes to implement document-level access control and collection-level access control. We have a fulfillment company that is made up of partners and employees. Partners have items they want the fulfillment company to fill, and the employees fulfill the orders of their partner's items.

For these examples, we have three collections:

Items: The items that the partners want the fulfillment company to ship.

Orders: The orders that the fulfillment company has to fill.

Employees: List of all the fulfillment employees.

And the fulfillment company wants to provide some queries that their partners and employees can use:

Partners can only see their items, but can't update them.

Partners can see their orders and update orders when filled.

Store employees can see all items and orders, but can’t update them.

Every partner has a unique API key. This key is used to authenticate them and restrict access to their information. Employees use their credentials for authentication and access control.

Partner #1: ACME Company -- API Key: acme_key.897b6e...

Partner #2: General Hardware Store - API Key: generalhw.28bc97…

Each API Key is being assigned a partner attribute:

In this section, we are creating four Query Workers to implement the queries mentioned above.

Partners can only see their items, but can't update them

User and API attributes are powerful tools to help with access control decisions on queries. Whether you are trying to limit documents or collections – or limit the viewing or modification of data – user and API attributes can address your requirements. After viewing these examples, you can quickly get started and apply attributes within your own GDN — based on your access control requirements. If you are new to Macrometa and would like to try out User Access Control or other features, 

 and see everything you can do in one platform.

Four Ways To Establish User Access Control With Macrometa

BAM! Simplifying Complex Event Processing

, we took a look at the high-level architecture of our log analytics dashboard example application that we developed with our partner 

. In this article, we look at the Stream Workers that are used for 

Macrometa provides developers with a simple way of performing 

 using Stream Workers. Macrometa Stream Workers gives you “superpowers” that allow you to transform data from one data type to another, enrich data by combining it with other databases or data services, aggregate streams, clean and filter data, and more.

 are written in Macrometa Stream QL. This domain-specific language combines the simple query language of SQL with declarative stream, source, and sink definitions. It also supports authoring arbitrary helper functions in Javascript.

 of the Stream Workers code in the Fastly log analytics demo application. We go over code patterns used in these Stream Worker examples, and you can view our documentation for more source code details.

In our Fastly log processing application, we use three stream workers. One sends HTTP requests to the Fastly service every three seconds to generate logs to analyze and the other two are responsible for reading and aggregating those Fastly web logs. For more information about the architecture of this log processing application, see the 

This Stream Worker uses trigger to make three HTTP requests to the Fastly service every three seconds.

Annotation to name the stream worker application:

 that runs the stream worker every three seconds. The stream worker will make three HTTP requests to the Fastly service to populate the Macrometa

FastlyGetLogGenerator, FastlyPostLogGenerator, FastlyPutLogGenerator, and FastlyInitCollectionGenerator streams.

The following stanza creates the stream definition for FastlyGetLogGenerator. There is a 

 for the stream, created using a sink annotation indicating that this is an HTTP sink along with a map annotation that defines the sink output format, in this case JSON. A sink, in the context of stream processing, is used to pass transformed data downstream for storage or additional processing.

Additional streams are defined for other HTTP methods for which we want to generate Fastly logs.

The Stream QL statement below retrieves a URL to contact from the FastLogGeneratorTrigger collection and inserts it into the FastlyGetLogGenerator stream (explained above). This will cause the FastlyGetLogGenerator stream to connect to the provided URL using the HTTP GET method.

Stream Worker fastly-http-request-worker (

This Stream Worker reads fastly logs from `logs` collection and performs reordering for out-of-order logs. Also, based on log timestamp, it determines the 10 second window and adds that window timestamp as an additional field.

Stream Worker fastly-http-request-stats-1m-worker (

This Stream Worker reads logs from `fastly-intermediate-stream` and performs aggregation within a 10-second window. It performs aggregation for response_status, url, response_body_size, latency.

 writes additional aggregates and summary data to Macrometa collections. Having read this article, the rest of the code should be straightforward to understand.

The Fastly log analytics application only scratches the surface of what can be accomplished with Macrometa Stream Workers. A rich selection of functions are available in Stream QL to facilitate Complex Event Processing.

. If you are new to Macrometa and would like to build your own superhero adventure, request a 

, and see everything you can do in one platform.

In our next blog in the series, we will be covering the use of Macrometa Query Workers in the log analytics application.

Using Macrometa Stream Workers In A Real-Time Log Analytics Dashboard

The code for this trading bot is open source. You can access it here: 

https://github.com/Macrometacorp/tutorial-cryptotrading

If you're going to try this project or customize it for your own strategy, you're going to need a Macrometa account. Take a minute to sign up for a free account by clicking on the button below.

I've been fortunate to have had a professional life that has exposed me to many experiences. I've not just tried my hand at software architecture and engineering, but also have been a full time professional quantitative trader for many years. The majority of my trading was with Index and Currency futures on Chicago Mercantile Exchange (CME), as well as trading US equities.

The general excitement surrounding cryptocurrencies piqued my interest. There are ~267 cryptocurrency trading exchanges spread around the world with probably around 8-10 exchanges with enough liquidity and reputation that one might consider working with them. Given the number of exchanges and volatility of cryptocurrencies, I wanted to see if it is feasible to build a trading bot that can trade on various exchanges doing something like exchange arbitrage.

An exchange arbitrage is basically a trading strategy based on the differences between the price of the cryptocurrency at different exchanges. Generally these opportunities open up if there is a price discrepancy and the discrepancy accumulates over time to finally become a significant amount like 1–3% at the cryptocurrency market. These arbitrage opportunities could exist for merely seconds, so an arbitrage trader would have to search for the best opportunities constantly and then implement them whenever possible.

Implementing arbitrage strategies in the crypto market is not that easy though, and involves a number of technical complexities. Besides having deposits in multiple major exchanges, on the technical side it involves the ability to monitor arbitrage opportunities simultaneously, making trades locally at each exchange with minimum delay and timely visibility & interactions between the trading agents distributed globally. The challenge requires big dollars and a team with the necessary technical expertise to build this type of distributed infrastructure.

Real-time crypto price data with Macrometa

 that offers a decentralized, distributed database, 

 engine and compute platform that runs across 175 global edge regions. Apps, APIs and web services written on Macrometa are automatically deployed globally and handle requests from the closest edge location relative to the request with local read-write latencies (i.e. very low latencies).

Macrometa offers a range of ready-made capabilities that enable an application like a crypto trading bot to be built quickly and easily. We will use the following features of Macrometa in this tutorial:


, decentralized and geo distributed developer platform that provides a database, a stream engine and stream processor and a function/container runtime.

 for storing trade data and sharing all trade related metadata and state between the decentralized, stateless bots and components

We will use geo distributed streams for publishing real-time price data from each exchange. Each bot subscribes to the stream and makes a decision on whether to buy or sell

Finally we will use real-time updates from the global database to automatically notify the clients/bots/subscribers of changes to the database so they can modify their trading strategy

So with the features listed above, I set out to see if Macrometa can be used to build a crypto trading bot to trade locally in multiple exchanges while providing global visibility & communication.

Some disclaimers before we proceed further - the trading strategy is an example of the capabilities of the Macrometa platform. Also, at present I do not trade cryptocurrencies. Trading involves significant risks, so always perform your due diligence. Please remember this post is about how to use the Macrometa platform and not about how or what to trade. T

his post is not to be used as trading advice or financial guidance.

For this tutorial, lets use a simple trend trading strategy:

The trading bot will run this strategy with:

Please note that the trading strategy is a hypothetical example and does not take into account slippage, spreads, position sizing, account balances etc. Feel free to enhance the strategy and let me know what the results look like. The code for this trading bot is open source.

The Crypto Arbitrage Trading Bot in Action

You can access the GUI for the single page app here - 

Clicking on above link should show something like what you see below.

For now go with defaults and click Confirm.

Once you click Confirm, the next screen will ask you to select the region for the dashboard to connect to. Currently this demo is running in 4 regions. Pick a region that is closest to you geographically.

Once you click Confirm, you will see a realtime dashboard like what is seen below.

Each of the charts in above picture represent the cryptocurrency pair quotes, its moving average and the exchange the quotes are from. This is served from the geo-replicated streams I mentioned before. The bot uses 1 geo-replicated stream per currency pair and exchange.

The bottom panel shows the trades made by the bot at each of the exchanges in that region. The panel is updated in realtime as the trades are made. You can use the filter box to filter the trades.

Now that you have an idea of what the dashboard looks like, it is time to dive into the nitty-gritty of the trading bot and dashboard code.

The trading bot uses geo-replicated streams in Macrometa's platform to publish local cryptocurrency quotes & moving averages in each region, and subscribe to the quotes and moving averages from other regions. Geo-replicated streams enable you to publish in any region and subscribe in any other region.

Similarly it uses the real-time database features of Macrometa to record trades locally in each region which are then automatically geo-replicated globally and visible to all regions in real-time. Most databases are single region and pull based in nature. Macrometa's database is geo-distributed and enables both pull and push based updates. The idea is each trading bot subscribe for updates on the trades collection (table) in each region to get updates in real-time.

Finally, the GUI is a small single page app that you can connect locally in any region and subscribe to the geo-replicated streams and trades collection to get a global view in realtime.

Building the global trading bot and dashboard wasn't as difficult as you'd think.The platform abstracts many of the complexities associated with distributed systems. As a developer, you can focus more on writing apps talking to a local system (for example, the trading bot & dashboard).

The source code is available on GitHub. You can access it here: 

Originally the trading bot was developed to run in 3 geo-locations (i.e., 3 instances) where each instance connects to one exchange locally, does the trades and publishes on respective geo-replicated streams and database. To make demos easier, we subsequently changed the code so that a single instance of the bot connects to all 3 exchanges and does the trades, while still publishing on respective geo-replicated streams.

The trading bot was developed in python using pyC8 driver (Macrometa's python driver). To make demos easier we changed the code to javascript using jsC8 (Macrometa's javascript driver).

 the details as to how to compile and run the trading bot and dashboard locally as well as via S3. 

This Crypto currency trading demo has two parts:

A UI dashboard application written in ReactJS

The node application has four main files:

 - Contains the Macrometa credentials to use for the demo.

 - The initialization work for jsc8 and streams is in this file.

 - This file gets the latest values from different exchanges and publishes them to their respective geo-replicated streams. It uses 

 library to connect to various exchanges.

 - This file subscribes to the streams, calculates the moving average to respective geo-replicated streams. Also, it triggers Buy and Sell trades and records them in the trades collection.

The demo creates two geo-replicated streams for each cryptocurrency pair it trades:

BTC/USD — cryto-trader-quotes-USD and crypto-tader-quotes-avg-USD streams

BTC/EUR — cryto-trader-quotes-EUR and crypto-tader-quotes-avg-EUR streams

BTC/JPY — cryto-trader-quotes-JPY and crypto-tader-quotes-avg-JPY streams

computes and publishes moving average to geo-replicated moving average stream

inserts a record into the trades collection.

Next are method inserts for the trade into the database feature of Macrometa's data platform.

The single page application (i.e., dashboard) subscribes to all the geo-replicated streams as well as to the trades collection to display the charts and trades in the dashboard.

This post became a lot longer than anticipated, so we're skipping the code explanation for the dashboard. Savvy javascript developers should be able to understand by looking at the code directly. You can find code for this in the below mentioned directory.

I hope you enjoyed the tutorial and can use it as a sample to write your own trading bots. Please feel free to let me know what you built and also any comments or feedback.

Finally, some unsolicited advice as a way of saying thanks for reading all the way. To me, a good trader is someone who gets checks from their broker regularly. Unfortunately for most people the checks go only in the other direction, i.e. from them to the broker. If you fall into this later category, ignore most of the stuff you read about trading. Instead, pick one simple strategy, stick to it, and paper trade until you are consistently profitable. That will be an education in patience, discipline, skill and trading mindset. Afterwards, trade with small money until you overcome the psychological challenges associated with real money on the line. That summarizes my two decades of trading experience part-time and full-time.

The code for this arbitrage trading bot is open source. You can access it here: 

How To Build A Crypto Arbitrage Trading Bot

One of the historical challenges of building real-time, distributed applications has been state. The missing piece is state – the contents of its memory that allow the application to run. Serverless has always been implied as a stateless process. So is the term "stateful serverless" a contradiction? Not anymore.

Macrometa's Justin Johnson and Redmonk analyst Kelly Fitzpatrick discuss how to build a real-time distributed application on Macrometa’s stateful serverless platform. Check out the demo below.

Build geo-distributed apps with the Macrometa CLI

What Is Stateful Serverless?

I originally wrote this in 2019, and I'm amazed at how much things have changed in just 3 years, from a technological and societal point of view. The cloud is going through a great generational shift before our eyes.

Put simply, we're now entering a new phase of the cloud where its architecture actually prevents it from solving emerging problems with data processing. The shift is from seeing and using data like its a historical record (seeing the world in the past tense), to seeing and using data as a dynamic set of events happening continuously (seeing the world in the present tense).

The cloud is built on the model of centralization: collect data from everywhere and centralize it in one big pile so that we can do something useful with it. But moving that data takes time and energy – the time cost often is latency and the energy cost is bandwidth. But there is nearly 2.5 quintillion bytes of data generated every day. By 2025, some estimate there will be 

 – all generating unbelievable amounts of data, much of it requiring real-time analysis. At this rate, the world will essentially be priced out of the centralized model.

Data needs to be ingested, processed, and analyzed where it is generated. Faster data processing requires it.

Data has time value - it tends to decay in value over time if not acted on. Data has location value when seen through the lens of ownership, privacy and data regulation. Data has concurrency value in situations where it's changing fast because of the actions of a lot of people on a common piece of data (like in auctions where the price value of something changes quickly), and finally data has actuation value - its only useful if acted on immediately (like in placing a buy order or sell order with a stock).

So take the types of data that have a mix of time, location, concurrency, and actuation value and move it to a centralized cloud model and you quickly realize the folly. The data becomes less valuable by the time it's moved and stored, it loses context by being moved, it can't be changed as quickly because all the moving around gets in the way of changing it, and finally by the time you act on it - it might be too late (your order doesn't fill in stock trades costing you profits). Therefore something has to change or evolve, that makes new assumptions about the real-time world in order for it to work. Fast data and the centralized cloud are like a race car meeting a wall at peak speeds. Lots of projectile debris and a wreck of twisted metal and rubber.

So how then do we solve this problem of fast data?

Geo-distributed fast data processing is the answer, edge computing is the architecture

Centralized cloud is the architecture of 

, but Fast Data needs its own architecture - an architecture that embraces distribution and decentralization. When we place data processing closer to the producers and consumers of data, we remove the cost of time and energy to move the data to a centralized location. The added step of analyzing this data in some central cloud is no longer necessary – we have what we need to do this at the edge. Centralized cloud at this point is a legacy system that we use despite its best efforts to stall or hinder us.

An edge architecture is not an extension of what worked in the cloud, it has to be built on the philosophy, physics, and mathematics of geo-distribution and latency. You can't simply take the primitives of data processing on the cloud-like eventually consistent object storage (S3 on AWS), clustered file storage (EBS, EFS or even lustre for that matter), high coordination overhead using consensus protocols (looking at you Paxos and Raft), and put it on the edge and expect it to work at scale. None of these systems were designed to provide high levels of accuracy (database consistency) at rapid rates of data change across large distances and a large number of locations.

Will the Edge Native Approach for Fast Data problems please stand up?

Cloud native was designed to work with the centralized cloud. We can't simply apply cloud native to our new edge architecture and expect it to work. This is a half measure. A real 

 architecture is one that is coordination free and provides high levels of accuracy on rapid rates of data change. Edge native allows developers to work with data as events – not mere state changes. This edge native architecture is event-driven, reactive to the real world and, most importantly, geo-distributed. The real edge is a brand new architecture, built on purpose specific components that solve the problems of dynamic network behavior, accurate data synchronization without the use of consensus.

The edge is a real opportunity to solve hard problems that the cloud cannot or will not solve. The edge is a hot and exciting space for new ideas, startups, and breakthrough business models. And we can expect that along with all the new interest will come inevitably, the great hordes of legacy tools vendors - rebranding their old school big data, storage and container products/services as "Now available with exciting new edge capabilities!"

Folks, we've seen that movie before. Every on-prem system vendor suddenly became cloud native with a Powerpoint slide, remember? The bar for edge computing is pretty damn high and it's not going to happen on the old cloud platforms.

Macrometa - the planet scaling, geo-distributed, fast data cloud

Macrometa provides the programming primitives for building low latency, request-response and event-driven applications for fast data.

A Global Data Network - cross region, multicloud by default with 175+ PoP locations around the world

A modern, NoSQL streaming database offering Key-Value, Documents, 

Strong and adaptive distributed consistency (accuracy) to handle various distributed concurrency scenarios

A compute runtime that lets you write event-driven code as pipelines that run with data locality across a set of user-defined locations.

Macrometa is first and foremost a platform for developers

 - its meant for people who want to solve the type of fast data problems that only edge architecture can solve. In building Macrometa - we have focused on providing the right abstractions to hide the complexity of distributed programming, accuracy and correctness. You don't need to know distributed databases or concurrent programming to write real-time, event-driven apps and APIs with Macrometa. You simply consume our APIs and let our platform do the orchestration, distribution, and scheduling of your data and code. 

Building applications the edge native way

The Great Shift - Edge Computing & Faster Data Processing

Real-time log analysis is complex and can require a huge amount of resources and tools. Having visibility of all these tools is an added challenge. At Macrometa, we give developers superpowers because we offer data storage and processing in one friendly interface. Tired of Kinesis or Splunk, or just want to explore new solutions in real-time log analytics? Macrometa and Fastly to the rescue!

Let’s talk about the technical architecture used to build a dashboard for real-time log analytics. This log analytics dashboard is part of a 

 that Macrometa recently developed with our partner 

The log analytics dashboard is composed of our 

 and Query Workers, a log capture service and a React JavaScript web app all built on Macrometa’s Global Data Network (GDN) and Fastly’s Compute@Edge. It leverages Macrometa’s pub/sub, 

, and database capabilities to ingest web logs and process them to produce real-time charts and dashboards.

The volume of logs produced by a streaming service like Netflix or Disney+ would challenge anyone tasked with creating and supporting an analytics dashboard. They may end up just hoping a superhero will swoop in and save the day. Not all heroes wear capes and Macrometa is here to save the day!

Macrometa’s platform combines a performant database with stream processing capabilities for Complex Event Processing along with a globally-distributed architecture. This allows us to process and aggregate large volumes of log data in a single bound from across the internet with very low latency and a consistent database.

This blog will describe the architecture of the log analytics application and explain the various components of the solution. We'll also give you access to the GitHub repo so you can try it our for yourself!

We have published the example log processing application code on 

 and the repo’s README includes step-by-step instructions on how to install, configure, and run the demo both locally and on Fastly’s Compute@Edge infrastructure. The application consists of Stream Workers, Query Workers, Macrometa collections, a log capture service, and a web dashboard written in React.

 is Macrometa’s stream processing framework and can be used for data processing tasks including transformation, enrichment, and aggregation.

In our Fastly log processing application, we use three stream workers. One sends HTTP requests to the Fastly service every three seconds to generate logs to analyze and the other two are responsible for reading and aggregating those Fastly web logs.

This Stream Worker uses triggers to make three HTTP requests to the Fastly service every three seconds.

This Stream worker reads Fastly logs from the `logs` collection and performs reordering for out of order logs. Based on the log timestamp, it determines the 10 second window and adds that window timestamp as an additional field.

Stream Worker Fastly-http-request-stats-1m-worker

This Stream worker reads logs from the `fastly-intermediate-stream` collection and performs windowing aggregation. It performs aggregation for response_status, url, response_body_size, and latency.

Query Workers allows developers to store parameterized Macrometa queries within the service and access them via a REST endpoint. Developers consuming data from Macrometa can thus access the data through this REST endpoint without having to provide the actual query, allowing a separation of business logic from presentation code.

Our log monitoring application uses the following Query Workers:

The dashboard in the React application is populated with the results of these queries.

Fastly pushes web logs to a Python application that inserts them into a Macrometa collection using the Macrometa Python driver. Once in the Macrometa stream collection, Stream Workers process and aggregate the logs.

The dashboard application is written in the React JavaScript framework. The react-app/src/app/services directory contains service methods that communicate with the Macrometa Query Workers to provide data for the log analytics dashboard. These methods are using the jsC8 Javascript Macrometa driver.

We'll go into more details on Stream Workers and Query Workers in future blogs. 

Fastly partners with Macrometa for Partner Ecosystem

Build A Real-Time Log Analytics Dashboard With Fastly And Macrometa

Macrometa is a geo-distributed streaming database with built-in compute as a service to provide low latency reads and writes to users across the globe.

In this blog, let’s build a geo-distributed, 

 Address Book application in 10 mins using our new Macrometa CLI.

Archive