dify-cloud-kit: Dify Cloud Kit: One Go Interface for Every Object Store

Most storage libraries help you talk to one provider. Dify Cloud Kit tries to make many providers feel like one system. That is a different kind of abstraction, and it matters any time your application needs to move between clouds, run locally, or keep tests from depending on real infrastructure.

The abstraction is the product

The heart of the repo is a compact Go interface named OSS. It gives callers a small set of operations, then hides the provider-specific machinery behind them. The point is not to expose every cloud feature. It is to make the common path boring.

type OSS interface {
    Save(key string, data []byte) error
    Load(key string) ([]byte, error)
    Exists(key string) (bool, error)
    State(key string) (OSSState, error)
    List(prefix string) ([]OSSPath, error)
    Delete(key string) error
    Type() string
}

That tiny surface area is doing a lot of work. The interface covers save, load, existence checks, metadata lookup, listing, and deletion. On paper that sounds ordinary. In practice, it lets the rest of an application stop caring whether the backend is S3, Azure Blob, GCS, Aliyun OSS, Tencent COS, Huawei OBS, Volcengine TOS, or local disk.

The repo is explicit about that goal in its own README. Dify Cloud Kit is a unified abstraction library for integrating various cloud object storage services in Go. It simplifies switching between providers and supports local testing and multi-cloud deployments.

Born inside Dify, then split out

This is not a greenfield toy. The code was extracted from Dify’s plugin daemon, which gives the repo a useful kind of credibility. It is young as a public package, but its design was shaped by real production needs, not just by the desire to build a clean abstraction for its own sake.

That lineage explains the tone of the code. The library does not behave like a framework that wants to own your application. It behaves like internal infrastructure that learned how to stand on its own. That makes the project interesting because it was not invented to look elegant. It was carved out because the pattern was already useful.

Where parity actually gets hard

The difficult part of cross-cloud storage is not that the APIs are unavailable. It is that they disagree about the shape of reality. One backend names things one way, another uses containers instead of buckets, and local disk adds its own filesystem semantics. Dify Cloud Kit smooths those differences most visibly in State and List.

State is the clearest example. Different providers expose size and modification time through different fields, different response objects, and sometimes awkward header parsing. The library collapses that into one OSSState result, so callers get a consistent answer instead of a provider-specific scavenger hunt.

List does a similar trick with paths. Cloud storage is key-value storage, but developers still think in prefixes, folders, and relative paths. The repo trims the noise, normalizes the output, and makes the local backend line up with the cloud backends closely enough to be useful in development and testing.

oss, err := factory.Load("s3", args)
if err != nil {
    return err
}

files, err := oss.List("images/")
if err != nil {
    return err
}

for _, file := range files {
    fmt.Println(file.Path)
}

How the factory turns strings into storage drivers

The factory layer is what makes the abstraction practical. You pass a string such as s3, aws-s3, or local, and the library resolves it to the right constructor. That is a small detail with big consequences. It means the caller can select a backend at runtime instead of hard-coding provider logic throughout the app.

Under the hood, that is just a map of constructors and a narrow interface. But the effect is bigger than the mechanism. A deployment can point at local disk during development, then switch to S3 or GCS later without changing the business code that reads and writes objects.

What it replaces, and where it fits

This is a narrower tool than a general cloud SDK, but that is exactly why it is interesting. Official SDKs solve provider access. MinIO’s client solves S3 compatibility. Dify Cloud Kit sits one layer higher and asks a different question: how do we keep the application code stable when the storage backend changes?

Trade-offs that matter

Every abstraction leaves something out. A shared interface can hide provider quirks, but it can also hide provider-specific power. That is the trade-off here. If your product needs deep, exotic features from one cloud, this library is not trying to be your final abstraction.

But if your real problem is portability, testing, or reducing the cost of switching backends, the trade-off is rational. The repo is small because its scope is disciplined. It does not try to make cloud storage simple in every possible sense. It tries to make it predictable in the ways most application code actually needs.