Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Warning

This page was translated from the original Japanese version by PLaMo Translate. The Japanese version is authoritative; the English translation may contain inaccuracies.

Using the Cache Service

This section describes the Cache Service provided by PFCP.

Note

This service is available at all sites except SR1. However, since it is configured using reserved nodes, it is unavailable to organizations that have not subscribed to reserved nodes.

What is the Cache Service?

The Cache Service is designed to accelerate access to frequently read data, such as datasets.

Its primary features include:

  • Shared access to the same cached data across multiple Pods
  • A simple read/write interface via HTTP (PUT, GET, etc.)
  • Authentication and authorization based on Kubernetes Namespaces
  • Capacity limits via per-bucket quotas

Cached data is managed in units called “buckets,” which correspond to Kubernetes Namespaces. Buckets can only be read from or written to by Pods in the same-named Namespace. Additionally, a quota (capacity limit) is set for each bucket to prevent any specific bucket from consuming excessive capacity.

Warning

This service is not persistent storage, and data persistence is not guaranteed. Furthermore, no encryption is used for either the communication channels or the storage. Please review the Important Notes before use.

Capacity Management via Quotas

The total capacity of cached data that can be held per bucket is managed via quotas. The capacity management algorithm uses Least Recently Used (LRU), where recently accessed cached data is prioritized for retention. If an upload would exceed a bucket’s quota, cached data is deleted in order of the oldest access time to free up the necessary capacity. If a single piece of cached data exceeds the quota, an error (413 Payload Too Large) is returned.

The default quota per bucket is 100 GiB. However, depending on the number of reserved nodes, the number of buckets, or the size of the cached data, LRU may delete cached data even if usage is below the quota. You can make effective use of the quota by creating cached data in smaller units (approximately a few MiB to several tens of MiB).

Note

Depending on the workload, “thrashing” may occur. Thrashing is a state where cached data is removed by LRU before it can be reused, leading to repeated cache misses, writes, and deletions. In this state, the hit rate decreases, and the caching benefits are not fully realized. In particular, for workloads that repeatedly read data in sequence in quantities exceeding the quota, the hit rate may drop to nearly 0% due to the nature of LRU. To avoid thrashing, it is important to allocate a quota appropriate for the workload. If you wish to change the quota, please contact support.

Usage

Dedicated buckets are provided for each Namespace. For example, the Namespace org-<organization-name>--foo can access the bucket of the same name, org-<organization-name>--foo, but other Namespaces cannot read from or write to this bucket. The following table shows access permissions for the bucket org-<organization-name>--foo:

User NamespaceAccess to Bucket org-<organization-name>--foo
org-<organization-name>--foo○
org-<organization-name>--bar×
org-<organization-name> (root namespace)×

Access to buckets is handled via authentication and authorization using Bound Service Account Tokens. Adding the label scs.preferred.jp/inject-config: "" to a Pod will automatically mount the Bearer token at /scs-secret/token.

kind: Pod
metadata:
  labels:
    scs.preferred.jp/inject-config: ""

When using Deployments or StatefulSets, add labels to spec.template.metadata.labels.

kind: Deployment
spec:
  template:
    metadata:
      labels:
        scs.preferred.jp/inject-config: ""

Note

Tokens have an expiration period of 3,600 seconds and are refreshed periodically. You must load the latest token for each request and include it in the HTTP header Authorization: Bearer <token>. PFIO, described below, has a built-in mechanism to automatically load the latest token, so no additional action is required.

v1 API

The v1 API endpoint is as follows:

http://scs.scs-system.svc.cluster.local/v1/objects/<bucket-name>/<cache-data-name>

For example, to interact with cache data cache-foo from the namespace org-<organization-name>--foo, access the following URL:

http://scs.scs-system.svc.cluster.local/v1/objects/org-<organization-name>--foo/cache-foo

The following HTTP methods are supported:

HTTP MethodOperation
PUTUpload cache data
PATCHUpdate scs-metadata
GETDownload cache data
HEADRetrieve cache data size and scs-metadata
DELETEDelete cache data

For each cache data item, you can store arbitrary metadata (up to 4,096 bytes) in the scs-metadata HTTP header. By embedding the update time or ID in scs-metadata, you can verify whether the cache data has been updated.

Note

For the PUT method, the cache data size must be specified in the Content-Length header. Additionally, Transfer-Encoding: chunked is not supported.

Uploading Cache Data and Updating scs-metadata

Use the PUT method to upload cache data. scs-metadata can be set using the PUT or PATCH methods.

$ echo "TEST" > test.txt
$ curl -H "Authorization: Bearer $(cat /scs-secret/token)" -X PUT http://scs.scs-system.svc.cluster.local/v1/objects/org-<organization-name>--foo/test --data-binary @test.txt
$ curl -H "Authorization: Bearer $(cat /scs-secret/token)" -X PATCH http://scs.scs-system.svc.cluster.local/v1/objects/org-<organization-name>--foo/test -H "scs-metadata: baz"

Viewing and Downloading Cache Data

To view metadata such as the size of the cache data, use the HEAD method.

$ curl -H "Authorization: Bearer $(cat /scs-secret/token)" -I http://scs.scs-system.svc.cluster.local/v1/objects/org-<organization-name>--foo/test
HTTP/1.1 200 OK
content-length: 5
content-type: application/octet-stream
scs-metadata: baz
...

From content-length: 5, you can see that the cache data size is 5 bytes, and that baz—updated via PATCH—is set in scs-metadata.

To download cache data, use the GET method.

$ curl -H "Authorization: Bearer $(cat /scs-secret/token)" http://scs.scs-system.svc.cluster.local/v1/objects/org-<organization-name>--foo/test
TEST

Deleting Cache Data

To delete cache data, use the DELETE method. If the quota is exceeded, cache data is automatically removed via LRU, but it can also be deleted explicitly.

$ curl -H "Authorization: Bearer $(cat /scs-secret/token)" -X DELETE http://scs.scs-system.svc.cluster.local/v1/objects/org-<organization-name>--foo/test
$ curl -H "Authorization: Bearer $(cat /scs-secret/token)" -I http://scs.scs-system.svc.cluster.local/v1/objects/org-<organization-name>--foo/test
HTTP/1.1 404 Not Found
...

Monitoring Usage via the Grafana Dashboard

In the Grafana Dashboard under “PFCP > Cache Service Usage,” you can check bucket usage across the organization (total number of cache data items, total capacity, cache hit rate, etc.).

Usage Examples

Below are some usage examples.

Accelerating Object Reads from S3-Compatible Storage Using PFIO

By using HTTPCachedFS from PFIO, you can transparently leverage the Cache Service to accelerate data reads from Amazon S3-compatible object storage (hereafter referred to as S3-compatible storage).

If cache data does not exist, PFIO reads the data from the original storage and automatically caches it. As a result, subsequent reads are faster. Since cache data can be shared across multiple Pods, this offers significant benefits when processing fixed datasets using Jobs or similar workflows.

#!/usr/bin/env python3
import os
import time
import uuid

import pfio.v2 as pfio_v2

endpoint = os.environ["AWS_ENDPOINT_URL"]
bucket = os.environ["S3_BUCKET"]
access_key = os.environ["AWS_ACCESS_KEY_ID"]
secret_key = os.environ["AWS_SECRET_ACCESS_KEY"]


underlay = pfio_v2.from_url(
    "s3://{}".format(bucket),
    endpoint=endpoint,
    aws_access_key_id=access_key,
    aws_secret_access_key=secret_key,
    create_bucket=True
)

cache_url = "http://scs.scs-system.svc.cluster.local/v1/objects/org-<organization-name>--foo"
token_path = "/scs-secret/token"

# If an object is updated, the cache data must be recreated.
# In this example, DATASET_VERSION is set as an environment variable
# to ensure that cache data is also recreated when the dataset is updated.
# If DATASET_VERSION is not set, a random value is used,
# so new cache data is created with each execution.
dataset_version = os.environ.get("DATASET_VERSION") or uuid.uuid4().hex

fs = pfio_v2.HTTPCachedFS(
    cache_url + "/cache-{}-{}".format(bucket, dataset_version), underlay,
    max_cache_size=10 * 1024 * 1024 * 1024,  # max 10 GiB cache
    bearer_token_path=token_path
)

key = "hello.txt"
payload = os.urandom(256 * 1024)  # 256 KiB

with fs.open(key, "wb") as f:
    f.write(payload)

read_times = []
for i in range(3):
    t0 = time.perf_counter()
    with fs.open(key, "rb") as f:
        data = f.read()
    read_times.append(time.perf_counter() - t0)
    assert data == payload
    print(f"read {i + 1}: {read_times[-1] * 1000:.1f} ms")

print(f"average read time: {sum(read_times) / len(read_times) * 1000:.1f} ms")

The first read takes longer because it retrieves the object from S3-compatible storage and stores it in the Cache Service, but subsequent reads are significantly faster.

$ python3 s3_example.py
read 1: 50.4 ms
read 2: 2.9 ms
read 3: 1.8 ms
average read time: 18.4 ms

If the dataset size exceeds the quota, performance may not improve due to the aforementioned thrashing. In such cases, consider splitting the dataset or adjusting the quota.

Tip

Range GET is also available, but read performance will degrade.

Accelerating Builds with ccache Integration

In CI/CD, integrating the http-storage feature of ccache with the Cache Service allows you to effectively leverage build caches. Since build caches can be shared across multiple Pods, this is particularly beneficial for use cases such as performing builds for every PR in GitHub Actions.

$ ccache --version
ccache version 4.9.1
Features: file-storage http-storage redis+unix-storage redis-storage
...

$ export CCACHE_MAXSIZE=10G
$ export CCACHE_REMOTE_ONLY=true
$ export CCACHE_REMOTE_STORAGE="http://scs.scs-system.svc.cluster.local/v1/objects/org-<organization-name>--foo/ccache|bearer-token=$(cat /scs-secret/token)"
$ ccache -s
Local storage:
  Cache size (GB): 0.0 / 10.0 ( 0.00%)
$ gcc -c hello.c
$ ccache -s
Cacheable calls:     1 /    1 (100.0%)
  Hits:              0 /    1 ( 0.00%)
    Direct:          0
    Preprocessed:    0
  Misses:            1 /    1 (100.0%)
Local storage:
  Cache size (GB): 0.0 / 10.0 ( 0.00%)
Remote storage:
  Hits:              0 /    1 ( 0.00%)
  Misses:            1 /    1 (100.0%)
$ gcc -c hello.c
$ ccache -s
Cacheable calls:     2 /    2 (100.0%)
  Hits:              1 /    2 (50.00%)
    Direct:          1 /    1 (100.0%)
    Preprocessed:    0 /    1 ( 0.00%)
  Misses:            1 /    2 (50.00%)
Local storage:
  Cache size (GB): 0.0 / 10.0 ( 0.00%)
Remote storage:
  Hits:              1 /    2 (50.00%)
  Misses:            1 /    2 (50.00%)

Note

In the example above, the token is configured directly in ccache, making it unsuitable for builds that exceed the token’s expiration period (3600 seconds). In such cases, a separate proxy must be introduced to insert the Authorization header.

Important Notes

Differences from Persistent Storage

Warning

This service is not persistent storage. Consequently, the following restrictions apply:

  • Data persistence is not guaranteed.
  • If cache data is updated before or after a failure, outdated cache data may be retrieved.

For practical use, we recommend implementing a mechanism to re-upload cache data if it is lost and a mechanism to assign new unique names to updated cache data.

Cache data is stored on the local storage of the contracted reserved node. Therefore, if the reserved node is cancelled or replaced, or if the local storage fails, the cache data will be lost.

Data Encryption

Warning

Encryption is not applied to either the communication channels or the storage. If encryption is required by law or other regulations, please implement it on the client side.

Maintenance

The Cache Service is configured using the contracted reserved node. Therefore, the following impacts may occur during maintenance:

  • Cache data may be deleted.
  • During maintenance involving node shutdowns, available capacity may temporarily decrease.
    • If you only have one contracted node, the Cache Service will be temporarily unavailable.