Analytics, Search, Cache, Backup, and Lifecycle

Goal

The previous section focused on source-of-truth databases. Real systems also need optimized copies and protection mechanisms around those databases.

Analytics, search, cache, backup, archive, lifecycle, encryption, and retention are not afterthoughts. They decide how the organization learns from data, keeps applications fast, recovers from failure, controls cost, and proves compliance.

The Short Version

Many data services are optimized copies with a specific job.

Use this map:

Source systems
  |
  +-- Search index        -> OpenSearch
  +-- Analytics store     -> Redshift or data lake patterns on S3
  +-- Cache               -> ElastiCache
  +-- Backup/recovery     -> AWS Backup, snapshots, point-in-time restore
  +-- Archive/lifecycle   -> S3 storage classes and lifecycle policies

The key question is not only "Which service?" It is also "How is this copy refreshed, owned, secured, monitored, rebuilt, and retired?"

Analytics: Redshift And Data Lake Patterns

Operational databases answer application questions: "Can this customer place this order?" "What is the current payment status?" "Which product record should the site display?"

Analytics systems answer business and aggregate questions: "Which regions grew fastest this quarter?" "Which products are abandoned in carts?" "What is the margin by channel?" "Which customer segments are changing?"

Amazon Redshift is AWS's managed data warehouse service for analytics at large scale. It is a good fit when teams need SQL-based analytical queries, business intelligence tools, dashboards, and reporting over curated data. S3 often appears beside Redshift as durable object storage for data lake patterns, raw extracts, staged files, historical data, and lifecycle-managed datasets.

The architecture decision is about workload separation. A checkout database should not carry every dashboard, ad hoc report, and historical aggregation. Analytics needs its own freshness expectations, data quality rules, transformation jobs, access controls, and cost model.

Search: OpenSearch As A Derived Index

Search is different from a database query. A product search feature may need text matching, relevance, filters, facets, ranking, typo tolerance, and exploration across many attributes. That is the job of a search index.

Amazon OpenSearch Service is a managed service for deploying and operating OpenSearch clusters. It is commonly used for search, log analytics, real-time application monitoring, and clickstream analysis.

Architecturally, OpenSearch should often be treated as a derived system:

• Source data changes in a product or order system.

• Events, streams, or batch jobs update the search index.

• The application queries the index for search experiences.

• If the index is wrong or lost, the team can rebuild it from source data.

That rebuild story matters. Without it, the search index slowly becomes an unofficial source of truth, and the team loses confidence in both systems.

Cache: ElastiCache For Hot Reads

A cache exists to make common reads faster or reduce load on a backing system. Amazon ElastiCache provides managed in-memory cache options using engines such as Valkey, Redis OSS, and Memcached.

Use caching when:

• The same data is read frequently.

• Low latency matters.

• The backing database should be protected from repeated reads.

• Temporary loss or rebuild of cached data is acceptable, or the design has a clear persistence strategy.

Caching is powerful, but it adds correctness questions. How is data invalidated? How stale can it be? What happens on cache miss? Is the cache storing sessions, computed responses, feature flags, or temporary read models? Which data must never exist only in the cache?

A cache can improve performance. It should not become a hidden database by accident.

Backup, Restore, Retention, and Lifecycle

Backup is not a checkbox. The real architecture questions are:

• What must be recoverable?

• How far back must recovery go?

• How quickly must recovery happen?

• Who can restore?

• How often is restore tested?

• How long must data be retained?

• When should data move to cheaper storage or be deleted?

AWS Backup can centralize and automate backup policies across supported AWS services. Individual services also have their own snapshot, backup, restore, replication, and point-in-time recovery capabilities. S3 lifecycle policies can transition objects across storage classes or expire them when retention ends.

Lifecycle is both a cost and governance decision. Hot data is expensive to keep hot forever. Cold data may be cheaper to retain but slower to access. Regulated data may need retention controls. Sensitive data may need deletion rules. Archives may need restore-time expectations, not just storage savings.

Example: Commerce Supporting Data

An e-commerce platform might use:

Each system needs an owner and a refresh or recovery rule. The product team may own source catalog data. The search platform may own index operations. The data team may own analytics transformations. The platform or operations team may own backup policies. Without that ownership, the architecture looks complete in a diagram but becomes unclear during incidents.

Common Mistakes

• Letting derived stores become untracked sources of truth.

• Designing backup without testing restore.

• Keeping all data hot because lifecycle was never designed.

• Adding a cache before understanding the slow query or access pattern.

• Running analytics directly against the production transactional database until customer-facing performance suffers.

• Forgetting encryption, access control, audit logs, and key ownership for copied data.

If Someone Asks You

Why might one application need search, analytics, cache, and backup services in addition to its main database?

A good answer is:

The main database owns operational state, but other services optimize different jobs. Search indexes support text search and exploration. Analytics stores support reporting and aggregate questions. Caches reduce latency and load for hot reads. Backup and archive systems support recovery, retention, and compliance. Each copy needs a clear job, owner, freshness rule, and rebuild or recovery story.

What You Should Remember

Derived data must have a job, an owner, and a rebuild or recovery story.

Questions to Check Your Understanding

1. How is a search index different from a transactional database?

2. Why should restore testing be part of architecture?

3. What can go wrong when a cache is treated as permanent data?