How to DWH with Python
رفتن به کانال در Telegram
Data Engineering practices, cases and implementation hints 👨💻 by @bryzgaloff
نمایش بیشتر461
مشترکین
اطلاعاتی وجود ندارد24 ساعت
اطلاعاتی وجود ندارد7 روز
اطلاعاتی وجود ندارد30 روز
در حال بارگیری داده...
کانالهای مشابه
هیچ دادهای
مشکلی وجود دارد؟ لطفاً صفحه را تازه کنید یا با مدیر پشتیبانی ما تماس بگیرید.
ابر برچسبها
هیچ دادهای
مشکلی وجود دارد؟ لطفاً صفحه را تازه کنید یا با مدیر پشتیبانی ما تماس بگیرید.
اشارات ورودی و خروجی
---
---
---
---
---
---
جذب مشترکین
آوریل '23
آوریل '23
+3
در 0 کانالها
مارس '23
+4
در 0 کانالها
Get PRO
فوریه '23
+3
در 0 کانالها
Get PRO
ژانویه '23
+3
در 0 کانالها
Get PRO
دسامبر '22
+6
در 0 کانالها
Get PRO
نوامبر '22
+11
در 0 کانالها
Get PRO
اکتبر '22
+23
در 0 کانالها
Get PRO
سپتامبر '22
+449
در 0 کانالها
| تاریخ | رشد مشترکین | اشارات | کانالها | |
| 29 آوریل | 0 | |||
| 28 آوریل | 0 | |||
| 27 آوریل | 0 | |||
| 26 آوریل | 0 | |||
| 25 آوریل | +1 | |||
| 24 آوریل | 0 | |||
| 23 آوریل | 0 | |||
| 22 آوریل | +1 | |||
| 21 آوریل | 0 | |||
| 20 آوریل | 0 | |||
| 19 آوریل | 0 | |||
| 18 آوریل | 0 | |||
| 17 آوریل | 0 | |||
| 16 آوریل | 0 | |||
| 15 آوریل | 0 | |||
| 14 آوریل | 0 | |||
| 13 آوریل | 0 | |||
| 12 آوریل | 0 | |||
| 11 آوریل | 0 | |||
| 10 آوریل | 0 | |||
| 09 آوریل | 0 | |||
| 08 آوریل | 0 | |||
| 07 آوریل | 0 | |||
| 06 آوریل | 0 | |||
| 05 آوریل | +1 | |||
| 04 آوریل | 0 | |||
| 03 آوریل | 0 | |||
| 02 آوریل | 0 | |||
| 01 آوریل | 0 |
پستهای کانال
Wow, have you known about this awesome hardware benchmarks page for ClickHouse? 🤩 See it: https://benchmark.clickhouse.com/hardware/
Results are contributed by ClickHouse users with various setups: from local laptops and bare metal VMs to cloud filesystems like AWS EFS.
In particular, I was interested in AWS EFS/EBS comparison: both are quite bad when compared to bare metal (which is no surprise 🤓) but with a huge advantage of EBS on cold runs 👍🏻
Hot runs EFS/EBS performance is comparable: both are about 6 times worse than bare metal.
Thus, both options are good for a quick MVP. EC2+EBS is a simpler setup while EFS can be attached to a disposable serveless ClickHouse container run as an ECS task.
| 2 | #youtube #briefly #ethereum Ethereum Data Analysis and Ingestion in AWS
🎥 YouTube talk + slides with transcription
What's inside:
— Building a realtime API for calculating tokens balances.
— Public vs own Ethereum nodes comparison.
— Support for other EVM and non-EVM blockchains.
Pick the most suitable format, same story:
— 🗒 Slides with a transcription for the video.
— ⚡️ A super-quick summary (2 minutes read).
— 📰 The original article, Medium. | 1 065 |
| 3 | #briefly #ethereum How to export a full Ethereum history into S3, efficiently
https://blockchain.works-hub.com/learn/how-to-export-a-full-ethereum-history-into-s3-efficiently-f37df
A brief summary of my original article about building a Data Platform for Ethereum:
— Which node to use: a free public one, a node provider, or run your own?
— Start querying right away: public BigQuery datasets with Ethereum data.
— How large is the dataset and how to process it cost-efficiently?
— Implementing a real-time Ethereum data ingestion.
Give it a chance if the original article is too long for you but you are interested in the best practices for Ethereum data engineering 😉 | 692 |
| 4 | #article #ethereum Exporting the full history of Ethereum into S3
https://medium.com/@tony.bryzgaloff/how-to-dump-full-ethereum-history-to-s3-296fb3ad175 (author: @bryzgaloff)
What's inside:
— BigQuery public datasets with Ethereum data: how to transfer to S3 quickly.
— Alternative approach: exporting data from a public Ethereum node. No need to run your own node!
— Processing uint256 with AWS Athena.
— Processing realtime updates from Ethereum.
— Best Data Engineering practices to process Ethereum data.
A short summary inside 👇 | 1 970 |
| 5 | Подготовил конспект статьи от Shopify о сетапе Airflow на 10 тысяч DAG'ов со 150 тысячами запусков в день. Сэкономит вам время на прочтении и поможет освежить в памяти в будущем.
#briefly #airflow Airflow: scaling out recommendations by Shopify
https://telegra.ph/Airflow-scaling-out-recommendations-by-Shopify-06-03
What's inside:
— Cloud Storage vs Network File System.
— Metadata retention policy.
— Manifest file.
— Consistent distribution of load.
— Concurrency management.
— Using different execution environments.
Origin: Lessons Learned From Running Apache Airflow at Scale | 0 |
