Видео

Hey youtube.com/@SeattleDataGuy, love your videos so far! Was curious if you'd like to add your insight into the following terms? Batch processing v stream processing OLTP v OLAP

Love hearing independent consulting best practices and what to expect! How do you approach accessing these clients' data and systems as a third party? Do they typically just give you a license (in the case of, say, Microsoft 365) as they would a W2 employee?

This guy is the best example when you spend 10 yrs of your professional life in "super-cheap-money-world" what happens, a smart kid with a very vague idea of the real world :)

Used it for years, I also tried the later 2.x version, I still don't like it, and I think there are better ways of architecting pipelines. But yeah I was amazed when I saw Airflow the first time, and it did solve a lot of problems, but I still think, it is a tool of the past. I hope I am wrong!

Not clearly explained, just bs .

I’m thinking of switching from full stack to data engineering . Any advice ??

We’re a young data team for a large organization. Biggest roadblocks for us are issues with data governance (“you can’t have or report on our data”), budget for tooling (“prove the value of the tool, then we can purchase it”), and cloud concerns (“all my data is on-prem. You can’t just put it in the cloud”)

This has happened to me. Now I’m leading a team of data scientists, engineers, analysts and migration specialists. I’ve had to learn so much so quick about strategy and people management. I’ve had to coach the people on my team to really empower and own their own tasks. At the beginning of being head of data I was taking on way too many “low level tasks”. Now I’m delegating and empowering. I still have alot to learn though.

I wonder how much Data God has in the Aether/Astral realm 🤔

If you're looking for help setting up your data team and strategy, then feel free to set-up a free consultation here - calendly.com/ben-rogojan/consultation

Just went thru this process with my company the past year. Great video. With us it went something like: - Where is all of our data - How are we doing reporting now - What are the shortcomings of existing reporting solutions - Do we need a warehouse (yes) - What warehouse do we pick - What ETL stack makes sense for our use case - What do we integrate in what order to maximize value and get adoption rolling Also, Having someone on the exec level champion the BI effort and really push it forward was huge for the thing to actually materialize.

I keep hearing from top CEOs that with Palantir we don't need teams anymore

Thank you for sharing

the de role should not exist, it should just be sde who also own data as a product. kind of lile front end, backend, thete will be a data focused engineer, that we can call data engineer. o wait

Hm. Makes me think that I should DM the data engineer that I vaguely know and have communicated with once or twice on Slack about what kind of work he does and if I would be able to work on low priority projects. Any recommended ice breakers?

Interested! can you please have special pricing for people in Africa. 50% reduction is good but our earnings are way too low probably 20x less than those in US or Europe. It becomes difficult for us to participate in this type of good courses. Any help! Thanks.

Hi! can you please share the full screen for what you have typed under the config_file?

I also find myself building stuff rather than analyzing business problems @@

Is there a service or scripting language that allows me to write relationships between tables/databases in a modern material design style?

Man, you really need to stop pitching your voice higher at the end of your sentences :(

Would love to see article reviews!

This is really awesome. Would love to see more of this!

thx for breaking it down

Don’t just be a task taker, be a strategic player.

E5 is where it's at 750 Million 😂

To me, one of the main benefit of Spark Structured Streaming is that you can easily switch between near real-time (micro batches) and scheduled batch processing without having to re-writing a single line of code. This is a very effective way of scaling up and down and balancing costs vs latency.

Is this really a good video? i feel like he just namedrops every techs

Hello ben, can we have a discussion please !

Great video! Thank you for sharing!

13:03 in Spark, we avoid Python UDFs like the plague because they're much slower than native Spark code. I wonder if the same is true for Flink, given that it also runs on JVMs. A quick Google search indicates that vectorized UDFs are a thing in Flink too, so I assume the same limitations apply

I’d watch if you did live article reviews!

Recommend me some books as I only have an idea of python and sql so..which book best for me as a beginner in data engineering field

Great video thanks!

gotta make things complex otherwise we wouldnt get paid as much. i half joke. we dont make it complex it's just situations are inherently complex

Sound of transition is very loud

Better crossover than a batman & Iron man movie.

Minor tip. I’d recommend removing the very high pitch transitions from your videos. I thought my tinnitus was acting up haha. Other than that great vid!

What are your thoughts on the new AWS Data Engineering Certification?

Thank you for this really great content! Which is the book you are referring to in your video? I like the structure much and am considering buying it. Thanks in advance!

I want to become a data scientist or engineer….my biology degree has never brought me financial security and I hope to be rich one day

Where do I find the theory behind the "types" of dimension tables?

Interesting use case about SCD2 but how in practice do we create these tables? I understand the importance and how useful is it to have a new row for each change but can’t get how to model it to make it work

I love learning about data engineering and overall cloud computing. Cloud is the future.

Thank you guys so much for your transparency- it is refreshing and I am more interested in working with you in the future as a result.

Are these salaries adjusted for 2024 tranches ? 😅🥲

awesome! I enjoy learning about these AI concepts in this hands-on practical way

Josue is the man! thank you both for the great conversation

Re: SQL Serverless…. Databricks now has job/workflow serverless which works with notebooks - a few limitations but most functionality is supported. I still use SQL all the time but that’s made the cluster start up penalty w notebooks way better

dissapointing to know that you don't answer your viewers' emails. Solid content though

Can you also make a video on the difference between DataBricks, Snowflake and Solix technologies