Hey everyone,
I am trying to build my first tech app, coming as a marketing person from the agency world this has been quite the change, but I've been learning the MEVN stack for about a year now.
A lot of tutorials talk about how X works, how MongoDB works, but I still don't see how it all comes together, and how everything interacts with one another. And I find myself stuck quite often.
Just wondering if this is common and what helped you overcome this?
I have many different ideas with different execution flows, and sometimes I'm not even sure if I am using a wrong stack for some tasks. Like for example server side video or image processing, would node even be optimal for this idfk...
And then it's like should I set up a separate server where the image/video processing happens whilst the main server is waiting for responses, probably, I guess. And how do I stich that together, what info do I keep in my DB.
Any tips, videos, or books would be appreciated. But talking to someone who has been through the struggles would be amazing.
Sounds like you are overthinking while over speeding.
There are so many questions to answer here but in short;
Don't sweat on stack, I googled MEVN. Whatever you do in MEVN is doable in any other web framework.
Start smaller, build a crud app. Build 10 crud apps. Once writing crud application become natural to you, implement authentication, implement 3 more times. Once it's become natural, implement roles, routes, role-based pages and role-based actions. Once it's done, go towards your goal. At this point you'll have an understanding of what needs to be stored in database and what was redundant to store.
Instead of how MongoDB works, knowing what is MongoDB what is a nosql/document based database is better. What alternatives are there, Relational Databases. Knowing the difference would help you choose what's best for the use case. If tutorials doesn't cover documentations are your friend and they usually guides you through basic setup on different use cases. If your use case is not there your are probably in a wrong place.
Make it work, then you'll optimize. There is no way for anyone to say x framework/library is better than whichever library you plan to use with node. My suggestion may have better benchmark performance on long form videos but that might not be what you need. To tackle this you need to test the framework by yourself without implementing it to your main project. Give it an input get your output if it works, it works go implement.
You may or may not need it depending on the load. You can get away with parallel programming or multi-threading without blocking user interaction.
At this point I don't think you need another book or video. You need to take a deep breath, realize it's not that easy and go back to fundamentals. Get better at it.
Fair enough mate, but speeding is somewhat necessary, annoyingly.
But I am currently doing small apps so I will just continue with that, I was just thinking maybe there's a birds eye view that would help me connect some dots as far as architecture goes.
To answer your main question you shouldn't be using a node server for image/video processing tasks. It's not good for CPU-intensive tasks ( check more on "node single-threaded event loop" ). I will suggest you go with "Go" as its multi-threading capability will help you to achieve your task with better performance. Or if you are a complete newbie then go with python.
Edit: Ah sorry I think my advice is a little specific to only one problem of your description. I will suggest you start from "freecodecamp" go there and start with all modules one by one just do them nothing else for now. You will come out with better understanding after completion.
I'm going to wade in with my opinions.
Since I'm the king of crap analogies, I'll bung in one for good measure. Let's say you wanted to clean up someone's yard. You'd first find out what type of shrubs, tree and debris you had to remove before you packed your vehicle with a chainsaw, shovel, cutters and gloves (the CSCG stack?). If you show up to find it has a big-ass tree, then you'll need a chainsaw. If it's just a bunch of leaves, you'd only need a rake and a pair of gloves. You'd be pretty annoyed if you spent all year watching chainsaw safety videos only to find a hedge trimmer would have done it.
Let the problem dictate the stack you choose. Don't pick a stack and then shoehorn your problem into the stack.
In Node.js you don't have access to these same APIs, but are instead offered a complex platform with numerous API calls for things like accessing IO, filesystem etc, network and so forth. Why does this matter? Well, I say it because most people assume that if they learn JavaScript in the browser they will somehow be able to take those client-side skills over to the backend. Although you'll have learned JS (the language), you will still have to learn two hosted environments which is a huge landscape to cover. This is something to keep in mind.
Another crap analogy. Let's say your first language was French and you decided to learn English (because everyone told you English is the universal language). So you spend a considerable amount of time learning the grammar, spelling and so forth. Your teacher decided the best way to learn would be to write blog posts about cheese so you spent a year doing this. Later, you decide you'd like to write a sci-fi novel. You sit down with your favourite coffee, but are you any better just because you spent a year writing cheese articles? Not really, you now have to spend another year learning about character plots, science and temporary twists. The only thing your cheese and sci-fi words have in common is that they are in the English language. I could go further with this analogy and talk about publishing (managing hosting) and so forth but I think you got the point.
An alternative solution might be to learn how to call server-side web APIs from JS in the browser and consume other people's backend APIs to get the job done. Another approach is to look into Firebase where you can build a complete app without needing any backend skills- but that's another topic. You mentioned in your post that you want to do some image/video processing. I would say that it's unlikely that you would need to write your own algorithms for image or video processing. Instead, you would use a ready-built API (AWS offers a lot of video and image processing APIs that'll do the heavy lifting for you).
That being said, if you just want to dive into your app and don't have the time or patience to learn SQL, you might want to consider choosing a hosted NoSQL database such as Firestore. Firestore comes with a free tier and has a very easy to understand and get started with data model. It can also be called from the client side, which can be appropriate for some apps (usually those that are highly UX driven).
My overall advice would be to start simple. Learn some HTML and some CSS (foundational pieces) first. Learn JavaScript (the language) first. Then take a look at the DOM in the browser, the fetch API, URL handling. Most importantly learn about HTTP (as in the protocol itself). Only then move onto a framework, if you really need to. I think Vue.js has a shallower learning curve that React IMO. I would recommend taking a look at Firebase, but stick to a few components only (Auth, Firestore and Functions using HTTP Callables). Functions will require some knowledge of Node.js but it'll be done wearing a straight jacket (i.e. you'll find these is a fixed way of producing and packing server side components that you can call directly from the client side).
Hopefully, some of this is helpful.
Hey, thanks for taking the time laying this all out. I took a few weeks to just try making crud apps to get better, learning a lot as I am going. Improved Vue skills and understanding 10x. Next taking the CRUD app and integrating Firebase Auth + a simple back end (express for example).
Could you please expand on 'using other peoples backend' - and how would that work? Any links that would help.
I completely get where you're coming from with the analogies, but the whole space was insanely confusing so I just decided to learn one flexible and modern stack and go from there. (The main hypothesis being: "If I understand one stack, it will be much easier to mix and match.)
Also I have like 20 ideas I'd like to test, so image/video processing is just one of many.
About Databases, it seems extremely simple, but I don't understand how people make it so complex + almost all apps have the same god damned name :D. A friend who's quite experienced recommended SQL Lite to get started with, what do you think? You didn't mention that SQL variant.
Could also be worthwhile learning FireStore.
But apologies if it was rude to not reply quickly, I just bunkered down for weeks on end learning and building stuff in the front end rather than talk about doing it.
By using other peoples backend tech I mean that sometimes you don’t need to even write your own database code. Instead you might called fetch /widgets to retrieve some resources. There are plenty of Web APIs offering backend as a service features in this manner.
I would say the recommendation of using SQLite is a good one, because it’s SQL based and there will be plenty of learning material. You will also be able to take what you learn to other database systems later if you need to scale (for example Postgres or CockroachDB). However, SQLite is a file based database system which means you’ll not be able to use it with ephemeral virtual machines. When you launch and kill virtual machine instances the disk drive usually dies with them. You would need to run a “pet” server and maintain it yourself.
By pet I mean, you’ll have to maintain it nurturing it as opposed to someone like Kubernetes which would be analogous to “cattle”. You wouldn’t SSH into an instance on a cluster of nodes in k8s.
Usually I prefer to run a managed Postgres server and then connect to it from any virtual machine wether a pet one I launched and SSHd into, or a ephemera instance launched on my behalf by a service such as Cloud Run. By keeping your database separate, you’ll be able to migrate your app to a larger virtual machine instance without having to bring down your entire app and copy across your website database SQLite file. But as I said, SQL is portable so it will be far easier to switch database systems later.
I'd honestly start by making a simple website w/ HTML, CSS and JavaScript, just to build confidence. Crawl before you walk. Also reference this blog (not mine) when you feel frustrated, it's completely normal to want to give up when learning to code. Don't.
https://medium.com/@nealkhosla/on-coders-block-72125ff9f50f