SmartCache: Context-aware Semantic Cache for Efficient Multi-turn LLM Inference

KeyDiff: Key Similarity-Based KV Cache Eviction for Long-Context LLM Inference in Resource-Constrained Environments