IP Functions are Generally Available, bringing high-performance network analytics to the Lakehouse

Imported from official source

  • Databricks now includes a family of native, built-in IP functions for parsing, validating, canonicalizing, and joining IPv4 and IPv6 addresses and CIDR blocks - no UDFs, no regex, no brittle bitwise math.
  • The first-class SQL, PySpark, and Scala functions are optimized in Photon so that demanding network workloads run in seconds instead of minutes.
    In head-to-head benchmarks, Databricks completed IP CIDR joins up to 3.1x faster and up to 6.4x cheaper than another leading cloud data warehouse.
  • Generally available today on Databricks Runtime 18.3+.
  • Every firewall, load balancer, VPN, CDN edge, DNS resolver, Kubernetes cluster, and application server all emit a stream of records keyed on one thing: an IP address. For a large enterprise, these streams collectively generate tens of billions of events a day and are the foundation of some of the most valuable analytics an organization runs, including threat detection, fraud investigation, and network observability. Historically, the industry treated these network observability use cases as specialized use cases that required a specialized stack, leading to silos, fragmented governance, and lock-in.

    That changes now. Today, IP Functions are available Generally Available. With this launch, IP address analytics becomes a first-class, high-performance SQL workload on the lakehouse, letting security teams parse, enrich, and analyze their highest-volume IP data alongside the rest of their analytics, under one governance model.

    Why network analytics used to be painful

    In the past, IP addresses were deceptively hard to handle in SQL. An IPv4 address looks like a string but behaves like a 32-bit integer; IPv6 is 128 bits. A CIDR block like 10.0.0.0/8 isn't a value at all - it's a range of 16 million addresses. Asking "is this IP inside that subnet?" is a range containment problem hiding behind a piece of text.

    This is an extract. The publication continues at the source.

    Read the original at the source: https://www.databricks.com/blog/ip-functions-are-generally-available-bringing-high-performance-network-analytics-lakehouse

    Officially imported this from Databricks’s own source and shows an extract. If you work there, claiming the profile and verifying the domain lets you choose to show the full text here.

    Provenance

    Organization
    Databricks — imported from official source
    Official source
    https://www.databricks.com/feed RSS
    Imported
    October 01, 2026 00:00
    Versions
    1 recorded
    Identity
    https://www.databricks.com/blog/ip-functions-are-generally-available-bringing-high-perf...

    Officially records where a publication came from, not whether it is true. Imported records are reproduced from an organization's own official source.