So enlighten me then, save me from my terrible hack that is working fine for me and tell me what it DOES have to do with. I thought S3 was a remote filesystem you can use, essentially Amazon’s proprietary version of webdav where you get a http bucket you can only access with aws proprietary tools. What’s the attraction? Clearly it seems like people love it, and I am getting dunked on for asking an honest question, which feels a bit unhealthy and unpleasant for the self-hosting community.
Am I supposed to be familiar with AWS infrastructure as a prerequisite for being here?
S3 is made for scaling. At an individual scale it doesn’t matter. At scale, S3 moves maintenance time away from managing individual issues about something not working or slow here and there towards configuring and maintaining a consistent piece of architecture
ok, to start with, if you need a POSIX interface to the filesystem, you already have an SSH connection to that server, and don’t need much stability across multiple clients, SSHFS may do just fine. For a homelab, that is likely the case.
now, if you’re hosting a web server that needs data distributed across drives/nodes, data redundancy, and the usage is primarily programmatic, closer to a CDN’s or machine learning pipeline than a single user browsing files; then you want an S3-compatible solution. The S3 API makes it easier to plug it into your application, while allowing you to migrate to a different one - which I’m actually currently doing for a MinIO deployment at work.
SSHFS isn’t POSIX compliant. It doesn’t support hard links, file locking, atomic renames, full support for changing file permissions, umasks, and probably other things.
S3 is designed for being used by applications via API, for example you can easily save and retrieve files from it even with a JavaScript application. It is much more difficult to do the same with sshfs
If instead you use it mounted on a computer, S3 is worse because each time you need to list its contents that’s an API request, if you have hundreds of thousands of files then it’s thousands of API reuqests
SSHFS is a hack and has nothing to do with the proposal of S3 compatible backends
So enlighten me then, save me from my terrible hack that is working fine for me and tell me what it DOES have to do with. I thought S3 was a remote filesystem you can use, essentially Amazon’s proprietary version of webdav where you get a http bucket you can only access with aws proprietary tools. What’s the attraction? Clearly it seems like people love it, and I am getting dunked on for asking an honest question, which feels a bit unhealthy and unpleasant for the self-hosting community.
Am I supposed to be familiar with AWS infrastructure as a prerequisite for being here?
S3 is made for scaling. At an individual scale it doesn’t matter. At scale, S3 moves maintenance time away from managing individual issues about something not working or slow here and there towards configuring and maintaining a consistent piece of architecture
ok, to start with, if you need a POSIX interface to the filesystem, you already have an SSH connection to that server, and don’t need much stability across multiple clients, SSHFS may do just fine. For a homelab, that is likely the case.
now, if you’re hosting a web server that needs data distributed across drives/nodes, data redundancy, and the usage is primarily programmatic, closer to a CDN’s or machine learning pipeline than a single user browsing files; then you want an S3-compatible solution. The S3 API makes it easier to plug it into your application, while allowing you to migrate to a different one - which I’m actually currently doing for a MinIO deployment at work.
SSHFS isn’t POSIX compliant. It doesn’t support hard links, file locking, atomic renames, full support for changing file permissions, umasks, and probably other things.
S3 is designed for being used by applications via API, for example you can easily save and retrieve files from it even with a JavaScript application. It is much more difficult to do the same with sshfs
If instead you use it mounted on a computer, S3 is worse because each time you need to list its contents that’s an API request, if you have hundreds of thousands of files then it’s thousands of API reuqests