Accessing Millions of Financial Data Points Using Snowflake Secure Data Sharing
Arcesium is a global financial technology and professional services firm, delivering solutions to some of the world’s most sophisticated financial institutions, including hedge funds, banks, institutional asset managers, and private equity firms.
Expertly designed to achieve a single source of truth throughout a client’s ecosystem, Arcesium’s cloud-native technology is built to systematize the most complex tasks. Arcesium supports more than $465 billion in assets with a staff of over 1,425 software engineering, accounting, operations, and treasury professionals.
CHALLENGE:
Directly Accessing Real-Time Financial Information
Arcesium’s technology is designed to optimize the daily and monthly middle- and back-office processes of institutional and alternative asset managers. The company processes millions of trades a day, generating profit and loss information for hundreds of thousands of holdings. This data is used daily by hedge funds, institutional asset managers, and all market segments in the financial services industry. Clients have full access to real-time information from any location and complete control over a highly sophisticated software solution.
Arcesium manages this real-time information in the AWS cloud. A key challenge for both Arcesium and its clients was that the data within Arcesium’s platform could not be queried or accessed directly. Instead, clients had to use Arcesium’s APIs to download the data, and then persist and process it on their local servers.
Arcesium quickly sought a way to enable its clients to directly access data as they would access a database. According to Dmitry Miller, Arcesium SVP, Head of Technical Relationship Management, “Clients wanted us to publish our data as a database because databases are convenient for reporting, analytics, and data warehousing. We wanted a cloud-native solution to provide this missing productivity tool to our client base.”
“When we first put Snowflake Secure Data Sharing into production, our product professionals ran a query for our client, and they were able to easily see their data. The client didn’t have to write any code; they didn’t have to call anyone; they didn’t have to provision any connectivity. It just worked”
-DMITRY MILLER, SVP, Product Management, Arcesium
SOLUTION:
Using Snowflake to Provide Fresh Data with High Performance
Arcesium chose to build its Data Warehouse product module on top of Snowflake, using the Snowflake Data Cloud to process its real-time business information. As data is produced during the day, it is persisted in Snowflake.
Data feeds come from dozens of sources, including market data vendors, prime brokers, custodians, and fund administrators.
According to Miller, “Data also comes directly from clients: their trade flow, transactions, and investment analytics. Our data sources are very diverse and require a substantial amount of data processing.”
To enable clients to access Arcesium data as a database, Arcesium uses Snowflake Secure Data Sharing. According to Miller, “With Snowflake, data sets get published directly to customers as a database, in addition to being available via APIs. For business data reporting data sets, the customer previously had to code a retrieval process and store data locally, and then solve for data freshness, performance, and technical support. With Snowflake, those challenges have been eliminated.”
In addition to the direct data access it provides to clients, Arcesium has a data analytics and visualization module that gives business users the ability to create visualizations from Arcesium’s data.
“By enabling access to our data via Snowflake, Arcesium has further enhanced our experience with their product, expanding what we’re able to do with our data. APIs have been a good mechanism for transactional operations; however, Snowflake Secure Data Sharing provides a better technology to publish reporting data sets to the consumers. It lowers the technical barriers to access public-cloud based platforms and allows a larger collection of our internal teams to leverage the data”
-THOMAS POLOGRUTO, Chief Data Architect & CTO of Liquid Markets and Data Science, Blackstone
RESULTS:
Direct Data Access and Increased Revenue from an Expanded Product Suite
Snowflake enabled Arcesium to transform its product offering, helping clients move to direct database access via Snowflake Secure Data Sharing. According to Miller, “The move to Snowflake has simplified clients’ ability to access data in our product offering. It makes their lives easier.”
Snowflake has enabled Arcesium to create and offer new modules to clients, expanding its data-based product suite to increase revenue.
“Snowflake unlocks new business possibilities for us, while also increasing our clients’ data flexibility and their overall satisfaction with our platform.”
-DMITRY MILLER, SVP, Product Management, Arcesium
FUTURE:
Joining Snowflake Data Marketplace and Expanding Snowflake Use
Going forward, Arcesium plans to provide and consume data in Snowflake Data Marketplace. According to Miller, “In our industry, data exchange processes are prevalent. They’re the bread and butter of what we do. We receive incoming data all day long, process it, and ship it off to the customer or third party working on behalf of the customer. With Snowflake Data Marketplace, we look forward to bidirectional flows of data with clients and with industry data providers.”
In addition, Miller and his team plan to move additional workloads from traditional databases to Snowflake and increase Snowflake use within the organization. Moving to the Snowflake Data Cloud will help Arcesium improve scalability and allow them to scale up or scale down as needed.
