One day, as you spend vast resources tracking and cutting and worrying about your AWS expenses, you’ll think “hey I could cut 100% of AWS costs by not using it!”.
Thinking about cutting AWS costs is your first step on the journey to never using it.
That’s great, until realise that you’re now spending money on infrastructure elsewhere instead.
I’m not going to pretend AWS is cost effective for every type of problem. But the comments here are overly simplistic.
Also, and more generally, I find it disappointing that when someone has made an open source tool to help the community, most of the comments are cheap attacks at the cost of running AWS. Poor etiquette guys.
> I’m not going to pretend AWS is cost effective for every type of problem. But the comments here are overly simplistic.
There overly simplistic comments from the "run from AWS" crowd as well as from the "just outsource everything to cloud" crowd. Nowadays going to cloud is still the easiest and safest bet if the company is not yours and it's big enough.
Run from AWS is also completely arbitrary. Someone building an expensive Rube Goldberg machine out of lambdas might hit cost problems way before someone essentially using AWS as a VPS provider with bare metal EC2
As data scientists, we usually don't get to choose. It's usually up to the hospital or digital lab's CISO to decide where the digitized slides are stored, and S3 is a fairly common option.
That being said, I plan to support more cloud platforms in the future, starting with GCP.
It's trivial to find and there are many alternatives.
Main problem is most support subset of the more advanced S3 features and often not all that big one. But if you just want to dump some backups in the cloud backblaze and other alternatives is cheaper
Frequently you have to couple the transactional state of the queue db and the app db, colocating them is the simplest way to achieve that without resorting to distributed transactions or patterns that involve orchestrated compensation actions.
that’s setting yourself up for trouble, imo. intermediate states solve this problem, and economically. for mature production system see temporal[0]. their magic sauce is good intermediate states.
i’m not associated with temporal, nor does the link above have any referrer nonsense in there. i don’t profit from referring to it here. in fact it may well be a household name in the hn community. that out of the way, it’s not wrong to point to a proper resource that can explain and demonstrate my argument better than a couple of words could. temporal is open source[0] so maybe a github link would have been more palatable?
solid_queue by default prefers you use a different db than app db, and will generate that out of the box (also by default with sqlite3, which, separate discussion) but makes it possible, and fairly smooth, to configure to use the same db.
Personally, I prefer the same db unless I were at a traffic scale where splitting them is necessary for load.
One advantage of same db is you can use db transaction control over enqueing jobs and app logic too, when they are dependent. But that's not the main advantage to me, I don't actually need that. I just prefer the simplicity, and as someone else said above, prefer not having to reconcile app db state with queue state if they are separate and only ONE goes down. Fewer moving parts are better in the apps I work on which are relatively small-scale, often "enterprise", etc.
Townscaper is a nice one like this; it has very few features or gameplay, just a sandbox tech demo for a very cool take on model synthesis / "wave function collapse" on an irregular grid. The game is mostly carried by the novelty of this mechanic (and also the pleasant art.)
I see you're not in the marketing department! We can do better by only considering "missed" as what wouldn't also be missed by a human: AI finds 71% of breast cancer*!
Depending how the costs of AI detection vs doctor, that genuinely might be enough to shift the math and be a net positive. If it is cheap enough to test 10x the current tested population, which would have lower, but non-zero rates of breast cancer, then[0] AI would result in more cancer detected and therefore more aggregate lives saved.
Given that every positive case needs to be verified by a doctor anyway because the patient has breast cancer, and every negative case has to be checked because it does a worse job than traditional methods... It only costs more.
Depends on the false positive rate. Hypothetically one can 'just' tune the model so false positives are low. This will increase false negatives but those are 'free' as they don't require follow ups. So long as the decrease in cost per real positive[0] goes down there's a benefit to be had.
[0] accounting for false positives, screening costs for true negatives, etc. etc.
> This will increase false negatives but those are 'free' as they don't require follow ups.
Increase in false negative rate significantly reduces survival rate and increases cost of treatment. We have huge multiplication factor here so decreasing false negative rate is the net positive option at relatively low rates.
> Depending how the costs of AI detection vs doctor, that genuinely might be enough to shift the math and be a net positive.
Based on my very superficial medical understanding, screening is already the cheap part. But every false-positive would lead to a doctor follow up at best and a biopsy at worst. Not to mention the significant psychological effects this has on a patient.
So I would counter that the potential increase of false-positive MRI scans could be enough to tip off the scale to make screening less useful
One day, as you spend vast resources tracking and cutting and worrying about your AWS expenses, you’ll think “hey I could cut 100% of AWS costs by not using it!”.
Thinking about cutting AWS costs is your first step on the journey to never using it.
reply