CH18-05: Security, Audit, Observability, and Operating Model
Goal
This section adds security, audit, observability, data classification, network boundaries, and ownership around enterprise integration.
The previous section handled change and failure at partner boundaries. This section asks a broader production question: can the integration be trusted, investigated, and operated by the right teams?
The durable idea is:
Integration is only trustworthy when every boundary is secure, observable, auditable, and owned.
The Short Version
Enterprise integration carries sensitive business movement: customer data, seller data, inventory movement, tax decisions, fraud signals, payment status, shipment updates, support case context, and finance records. A design that only moves data is incomplete. It must also control who can move data, which data is allowed to cross the boundary, what evidence is retained, how failures are detected, and who responds.
Security, audit, observability, and ownership are not separate add-ons. They are part of the integration architecture.
Operating Responsibility Map
Use this map for every meaningful integration path:
Integration path -> security control -> telemetry -> audit evidence -> owner -> runbook| Area | Design Question | Example Control |
|---|---|---|
| Identity | Which workload, partner, user, or system is calling? | Microsoft Entra ID, managed identity, certificates, partner credentials. |
| Authorization | What is the caller allowed to do or see? | API scopes, RBAC, policy checks, domain authorization rules. |
| Secret handling | How are credentials, certificates, and keys protected? | Azure Key Vault, managed identities, certificate rotation. |
| Network boundary | Which paths are public, private, hybrid, or partner-facing? | Private endpoints, firewall rules, virtual networks, gateway policies. |
| Data classification | What kind of data crosses this boundary? | Customer PII, payment-related data, seller data, tax records, support notes. |
| Observability | Can the team trace requests, messages, events, and failures? | Correlation IDs, logs, metrics, distributed traces, queue depth, retry counts. |
| Audit | Can the business prove what happened? | Immutable records where needed, request/response summaries, change history. |
| Ownership | Who responds and who owns the business meaning? | Platform runbook plus domain owner escalation. |
This map keeps production readiness visible. It also helps readers avoid a common weak answer: "We will monitor it" without saying what signals matter or who acts on them.
Security Controls for Integration
Security starts with identity. Internal workloads should avoid embedded credentials when a managed identity or service principal can provide stronger lifecycle control. Partner integrations may require certificates, OAuth flows, signed requests, API keys, or network allowlists depending on the partner and risk level.
API-facing integrations should enforce authentication and authorization before traffic reaches backend services. API Management can help centralize gateway policies such as rate limiting, token validation, request transformation, and logging. Backend services still need their own authorization for business rules. The gateway can prove who is calling; the domain service decides what the caller is allowed to do in business terms.
Messaging and event paths also need authorization. A producer should not be able to publish arbitrary messages to every topic. A consumer should not be able to read every queue or subscription. Limit access by workload identity and by integration responsibility.
Secrets, certificates, and connection strings should be stored and rotated through a controlled secret management process. Azure Key Vault is commonly used for storing and controlling access to secrets, keys, and certificates. The architecture should also define who can read secrets, who can rotate them, and how rotation is tested.
Data Classification and Privacy
Integration moves data between contexts. That makes classification critical.
Commerce integrations may carry:
Customer names, addresses, emails, and phone numbers.
Seller bank, tax, and business identity data.
Payment status and transaction references.
Fraud signals and risk scores.
Support case notes.
Inventory, supplier, pricing, and margin data.
Finance, invoice, settlement, and reconciliation records.
Each category can have different retention, masking, encryption, access, audit, and geographic requirements. The design should avoid copying sensitive data into every downstream system just because it is convenient. Send the minimum data needed for the integration's purpose. Mask or tokenize where appropriate. Retain audit evidence without exposing more customer or financial data than necessary.
Observability Signals
Integration observability must combine technical and business signals.
| Signal | Why It Matters |
|---|---|
| Request rate and latency | Shows whether APIs and partner calls are healthy and within expected behavior. |
| Error rate by partner and operation | Separates one partner incident from a general platform problem. |
| Queue depth and oldest message age | Shows whether durable work is backing up and how long customers may wait. |
| Retry count and dead-letter count | Reveals stuck work, bad data, partner failures, or broken deployments. |
| Event delivery and consumer lag | Shows whether subscribers are falling behind. |
| Correlation ID coverage | Lets teams trace a customer, order, seller, shipment, or invoice across systems. |
| Contract version usage | Helps teams plan deprecation and avoid hidden consumers. |
| Business outcome metrics | Connects technical health to checkout, fulfillment, support, finance, or seller impact. |
Azure Monitor can collect and analyze logs, metrics, traces, and alerts across Azure resources and applications. Application Insights and OpenTelemetry can help trace application paths. The important architecture habit is to design correlation and business identifiers into the integration flow from the start.
Audit Evidence
Audit is not the same as logging everything.
Logs help engineers troubleshoot. Audit evidence helps the business prove what happened. In enterprise commerce, audit may be needed for tax, finance, settlement, fraud investigation, partner dispute, customer complaint, compliance review, or operational accountability.
Good audit records answer:
What was requested?
Who or what requested it?
Which version of the contract was used?
What decision or response came back?
When did each step happen?
Which system owned the decision?
Was the work retried, replayed, corrected, or manually approved?
The design should be careful with sensitive payloads. Sometimes the audit record should store hashes, identifiers, summaries, or references rather than full request and response bodies.
Ownership Model
Enterprise integration needs both platform ownership and domain ownership.
| Owner | Owns | Should Not Own Alone |
|---|---|---|
| Central integration platform team | API gateway standards, shared messaging infrastructure, workflow platform, monitoring patterns, partner onboarding guardrails, reusable templates. | Business meaning of every event, message, and decision. |
| Domain team | Contract meaning, domain validation, business failure behavior, support interpretation, customer or seller outcome. | Shared platform reliability and cross-cutting integration standards. |
| Security and compliance | Identity standards, data classification, secret handling, audit requirements, access reviews. | Day-to-day business triage for every integration failure. |
| Operations or SRE | Alerting, incident response process, runbooks, service health, escalation coordination. | Domain-specific correction decisions without business owner involvement. |
The split should be visible in runbooks. When an order export fails, who checks the platform? Who checks the finance contract? Who decides whether to replay? Who communicates business impact? If the answer is "the integration team," the model is probably too vague.
Example: Tax Provider Failure During Checkout
Suppose a tax provider starts failing intermittently during checkout.
A production-ready design should provide:
Alerts for elevated failure rate, timeout rate, and checkout impact.
Correlation IDs from checkout request through tax workflow and provider call.
Logs showing provider response codes, contract version, and request category without exposing unnecessary sensitive data.
A fallback policy owned by commerce and finance, not invented by engineers during the incident.
A runbook that says whether checkout should retry, fail closed, estimate, queue for review, or block affected regions.
Audit records for tax decisions and later corrections.
Owner contacts for commerce, finance/tax, integration platform, and provider management.
This is what makes the integration operable. The technical call is only one part of the production system.
Common Mistakes
Building a central integration layer that owns everything and understands nothing. Shared platforms help, but business meaning still needs domain ownership.
Logging technical errors without business correlation. A stack trace is less useful if support cannot connect it to an order, customer, seller, shipment, or invoice.
Treating audit as a reporting export instead of an architecture requirement. Audit evidence must be designed into the path where decisions happen.
Storing sensitive payloads everywhere. Observability and audit should be useful without unnecessary data exposure.
If Someone Asks You
How do you operate enterprise integrations safely?
Design every integration path with identity, authorization, secret handling, network controls, data classification, observability, audit evidence, ownership, and runbooks. Use central integration capabilities for shared controls and consistency, but keep domain teams accountable for business meaning and failure decisions. Make sure telemetry can connect technical failures to customer, seller, inventory, payment, delivery, support, or finance outcomes.
What You Should Remember
Integration ownership must cover the technical path and the business meaning.
Questions to Check Your Understanding
What telemetry would help investigate a failed partner integration that affected customer orders?
Why does data classification matter before designing an integration path?
How should central integration teams and domain teams share ownership without creating gaps?