I was following official documentation of Offload Media Plugin to move my media library to AWS S3 and serve them from CloudFront. I created the bucket using the plugin settings and after setting up a CloudFront distribution, I wanted to enable Block All Public Access feature. But when I tried to do it from plugin settings the error "Could not change Block All Public Access status for bucket" popped up.

Then I when to S3 dashboard and was able to successfully enable it. But still Offload Media Plugin wasn't working as expected. I can still see the error popping up when I try to save the bucket. After digging a bit, I noticed that my Custom CNAME is redirecting to bucket endpoint. Further Research landed me on a AWS support document which included following text,

After you create an Amazon S3 bucket, it can take up to 24 hours for the bucket name to propagate across all AWS Regions. During this time, you might receive the "307 Temporary Redirect" response for requests to regional endpoints that aren't in the same Region as the S3 bucket.

So for me, waiting couple hours was the answer. If you ever run into this weird error. Take a rest and drink a coffee. There doesn't seems to be anything we can do but to wait for the propagation.