DATABASE DESIGN

Introducing Auto-Index Creation for Atlas Serverless Instances Read: https://www.mongodb.com/blog/post/introducing-auto-index-creation-atlas-serverless-instances

Data Governance for Building Generative AI Applications with MongoDB The text discusses the importance of data governance in building generative AI (GenAI) applications with MongoDB. It highlights the rapid evolution and adoption of GenAI tools and models, such as OpenAI's ChatGPT, Cohere's LLM, Google's Med-PaLM, and Microsoft's integration of GPT-4. It emphasizes the need for flexibility with data and the requirement for data governance to ensure data security, accuracy, privacy, availability, and usability. The text also mentions potential security risks in building GenAI applications, such as data security and privacy, intellectual property infringement, regulatory compliance, explainability, and AI hallucinations. It explains how MongoDB addresses these security risks through its developer data platform, MongoDB Atlas, which offers built-in security controls, encryption tools, regulatory compliance support, and regular security audits. The text concludes by discussing additional best practices for working with AI models and the importance of strong data governance in leveraging the power of GenAI tools. WOO Network faced challenges with the rapid growth of their business and the need to introduce a NoSQL database solution. They chose MongoDB Atlas on Google Cloud Platform to address the issues of performance degradation and slow data queries. MongoDB Atlas provided them with scalability, flexibility, and ease of operation. With features like auto scaling and sharding, WOO Network was able to ensure smooth transactions and handle large amounts of historical data. MongoDB Atlas also offered compatibility with different data formats and built-in management tools, reducing the burden on their development and operations teams. WOO Network plans to continue utilizing MongoDB Atlas for various data applications and aims to provide diversified services to their users. The integration of MongoDB Atlas has helped WOO Network improve performance, user experience, and operational efficiency. They intend to further collaborate with MongoDB to enhance their cryptocurrency trading platform. Pureinsights is a search and AI application specialist that can help shorten the planning and development cycle for customers. They specialize in building vector search applications and offer services to create and populate MongoDB Vector Search and UI/Client to search MongoDB Atlas. Pureinsights has expertise in application design, build, and managed services, and can help customers build and deploy next-generation vector search applications. They have a methodology for analyzing current applications called the Search Maturity Matrix, and offer managed services to maintain search applications for optimum performance. Pureinsights can help customers overcome the challenges of building vector search applications and accelerate time to production. Read: https://www.mongodb.com/blog/post/data-governance-building-generative-ai-applications-mongodb

Powering Vector Search Maturity in Retail with Pureinsights Read: https://www.mongodb.com/blog/post/powering-vector-search-maturity-retail-pureinsights

Переход с ETL на ELT ETL (Извлечение-Трансформация-Загрузка) и ELT (Извлечение-Загрузка-Трансформация) — два термина, которые часто используются в области инженерии данных, особенно в контексте захвата и преобразования данных. Хотя эти термины часто используются как взаимозаменяемые, они относятся к немного разным концепциям и имеют различные последствия для проектирования конвейера данных. В этом посте мы проясним определения процессов ETL и ELT, обозначим различия между ними и обсудим преимущества и недостатки, которые они предлагают инженерам и командам по работе с данными в целом. И самое главное, я опишу, как недавние изменения в формировании современных команд по работе с данными повлияли на ландшафт борьбы ETL против ELT. Понимание Извлечения (Extract), Загрузки (Load) и Трансформации (Transform) независимо друг от друга Главный вопрос при сравнении ETL и ELT, очевидно, последовательность выполнения шагов Извлечения, Загрузки и Трансформации в рамках данных. Читать: https://habr.com/ru/companies/itsumma/articles/780612/

WOO Network借重MongoDB Atlas優勢 持續優化WOO X平台服務能量 Read: https://www.mongodb.com/blog/post/woo-network-leverages-advantages-atlas-continuously-optimize-platform-service-energy-cn

New MariaDB Enterprise Server Releases 10.4-10.6 with Backported Features Read: https://mariadb.com/?p=38394

Записки оптимизатора 1С (Часть 5). Ускорение запросов с RLS в 1С системах Замахнемся сегодня на RLS. Обсуждать будем проблемы по нашему профилю, связанные с производительностью 1С:Предприятие. Но, в целом, этот материал может быть полезен и не только 1С-никам. Почему запросы с RLS очень часто такие долгие? Какие есть варианты их ускорить? Читать: https://habr.com/ru/companies/softpoint/articles/780340/

Expanding Monty’s Role at MariaDB Read: https://mariadb.com/?p=38389

d maintenance convenience, which affected production continuity. To address these challenges, SEWC partnered with MongoDB to build the next-generation Manufacturing Execution System (MEMO) data foundation. The MEMO system is built on a container private cloud platform and microservices architecture, providing flexibility, high availability, open access through APIs, and easy replication and scaling between factories. MongoDB's distributed document model database provides the flexibility needed for data modeling, high availability through a primary-secondary replication set architecture, and improved operational efficiency through MongoDB Ops Manager. By adopting MongoDB, SEWC was able to avoid the risk of system crashes, improve operational efficiency, and unleash productivity. As industrial intelligent scenarios become more prevalent, the rapid growth of data poses a significant challenge for enterprises. SEWC recognizes the urgent need to address the storage management and database pressures caused by the industrial data generated every moment. Most of the content is related to the advantages and benefits of using MongoDB for real-time data management in industrial scenarios. The traditional information systems allow downtime, which is not acceptable in industrial settings with increasing numbers of devices. MongoDB improves development flexibility, meets high availability requirements, and enables automated operations. The testimonial from a SEWC IT engineer highlights the flexibility and support provided by MongoDB for their MEMO system, ensuring continuous production and freeing up time for more valuable tasks. Read: https://www.mongodb.com/blog/post/navigating-indian-tech-landscape-mongodbs-path-success

Navigating the Indian Tech Landscape: MongoDB's Path to Success The article discusses MongoDB's strategy, challenges, and future opportunities in the Indian tech landscape. MongoDB has seen significant growth in India, with over 3,100 customers and a 40% year-on-year increase. The thriving startup ecosystem, enterprise transformation, and large developer pool in India present opportunities for MongoDB. The company attributes its success to technological innovation, a talented pool of employees, and a strong partner ecosystem. MongoDB emphasizes a collaborative team approach and a focus on recruiting individuals with a growth mindset and intellectual honesty. Looking ahead, MongoDB aims to recruit the right people, maintain its culture, and continue its focus on innovation, talent, and collaboration for long-term success in the Indian market. SEWC is an advanced factory that focuses on researching and producing SIMATIC industrial automation system products, including PLC (Programmable Logic Controller), HMI (Human Machine Interface), IPC (Industrial PC), IOT (Industrial Gateway), and industrial edge computing software products, to serve the Chinese and global industrial markets. With continuous improvements in team efficiency and technical capabilities, SEWC continuously iterates digital twins, data analysis, artificial intelligence, and automation technologies. In terms of product quality, SEWC has maintained a world-class level since its establishment, with a 99.999% process quality. In terms of product production, SEWC serves Chinese customers better through localized production and advanced production modes, helping China's industry achieve intelligent manufacturing transformation. Established ten years ago, SEWC has grown from scratch, using digital technology to unleash the unlimited potential of people and gradually become a leading manufacturing lighthouse factory and carbon-neutral factory for the future of digitization. The text also mentions the business challenges that SEWC faces. They need to break through the bottleneck of traditional relational databases and ensure continuous production. The digital transformation in the industry has created a collective change, with the emergence of digital technology in various industry chains. In the field of industrial manufacturing, the key to enterprises' resilience and competitive advantage lies in whether they can achieve close coordination between production elements and processes, efficient planning, and management of the entire production process through digital technology. SEWC, as Siemens' first mature digital factory in China, recognizes the importance of Manufacturing Execution Systems (MES). However, with the continuous and rapid changes in business, traditional MES becomes inadequate to meet the needs of enterprise digital and intelligent transformation. SEWC produces over 2,300 different products in a space of around 10,000 square meters, often with different production methods mixed together. This mixed-mode production generates new core requirements and gives rise to a new generation of MES. The previous generation MES system, SimaticIT, integrated various equipment and systems on the production line, facilitating production planning monitoring and execution, optimizing production processes, reducing production interruptions and resource waste. However, it was based on traditional relational databases, which required constant redefinition of data structures and was prone to crashes during parallel processing of massive data. Traditional relational databases also posed many challenges in system operation and maintenance, especially when secure maintenance was necessary, production had to be stopped to avoid data damage. Overall, the previous generation MES system faced a series of challenges in modern digital production applications, such as lack of flexibility, low development efficiency, single point of failure in databases, lack of high availability guarantees, lack of horizontal scalability, an[...]

MDM и CDP: различия систем. Как сделать выбор Любой компании, которая стремится сохранить конкурентоспособность на рынке, необходимо создание «золотой записи» (профиля) клиента во внутренних базах. Для этого многие используют системы управления мастер-данными (MDM, master data management), но сталкиваются с рядом проблем, другие – применяют CDP-платформы (Customer Data Platform), которые также имеют свои недостатки. А теперь представьте, если от каждого из решений можно было взять лишь лучшее, оставив за бортом все слабые стороны. Как это сделать – в новой статье CleverData под катом. Читать: https://habr.com/ru/companies/lanit/articles/776862/

Курс «PostgreSQL для начинающих»: #1 — Основы SQL Этим постом я запускаю публикацию расширенных транскриптов лекционного курса "PostgreSQL для начинающих", подготовленного мной в рамках "Школы backend-разработчика" в "Тензоре". В программе: рассказ об основах SQL, возможностях простых и сложных SELECT, анализ производительности запросов, разбор [не]эффективного применения индексов и особенностей работы транзакций и блокировок в этой СУБД. Курс не претендует на лавры "войти в айти", поэтому подразумевает наличие у слушателя опыта программирования или работы с другими СУБД, и, главное, желания самостоятельно изучать тему работы с PostgreSQL глубже. Для тех, кому комфортнее смотреть и слушать, а не читать - доступна видеозапись. Читать: https://habr.com/ru/companies/tensor/articles/779698/

MongoDB助力西门子数字化工厂构建下一代制造执行系统 MongoDB is collaborating with SEWC, a subsidiary of Siemens, to build the next-generation manufacturing execution system (MES) for SEWC's digital factory. SEWC, located in Chengdu, China, is Siemens' first digital factory established outside of Germany and is recognized as one of the world's most advanced factories. As SEWC continues to improve its team efficiency and technological capabilities, MongoDB's flexible and high-performance document model database is chosen to overcome the limitations of traditional relational databases and ensure continuous production. The new MES system, called MEMO, is built on MongoDB's distributed document model, providing flexibility, high availability, and scalability. MongoDB's features, such as the ability to handle different data structures, replica sets for high availability, and the MongoDB Ops Manager for efficient operations management, address the challenges faced by traditional MES systems and help SEWC achieve digital transformation and smart manufacturing. The text is discussing the use of MongoDB in different scenarios. In the first part, it talks about how MongoDB is used in a manufacturing company to improve flexibility, meet high availability requirements, and automate operations. The text also includes a testimonial from an IT engineer at the company who praises the benefits of MongoDB in their operations. In the second part, the text discusses how Samsung Knox, a mobile device security platform, is using MongoDB for its cloud solutions. It highlights the role of MongoDB in providing robust cluster management, high performance, and disaster recovery capabilities. The text also mentions the use of MongoDB University for educating developers and mentions plans for further collaboration with MongoDB. The third part of the text introduces Zelta.ai, a company that uses generative AI and MongoDB to prioritize product roadmaps based on customer feedback. It discusses the use of MongoDB for storing data and highlights the flexibility it provides for experimentation and easy schema migration. The text then briefly mentions Crewmate, a company that builds AI-powered communities, and Ada, a company that helps product companies support their customers through AI-driven automation, without providing many details. Overall, the text highlights the benefits of using MongoDB in different industries and scenarios, such as improving flexibility, high availability, automation, and efficient data storage and analysis. In this text, Raj Thaker, the CTO and Co-Founder of Crewmate, discusses the importance of MongoDB's flexible document schema for storing data of any structure. He highlights the benefits of Atlas Vector Search and the Building Generative AI Applications tutorial, which provide a blueprint for integrating a database, vector search, and real-time data pipelines. Thaker also praises the MongoDB Query API for processing and analyzing user engagement data without the need for a separate data warehouse. Like Zelta, Crewmate is part of MongoDB's AI Innovators program. Mike Gozzo, Ada's Chief Product and Technology Officer, explains how Ada uses MongoDB Atlas to power its AI-powered automations for customer service. He emphasizes the flexibility and scalability of MongoDB Atlas, allowing Ada to easily adapt and extend its data store as the company grows. Gozzo also mentions the ability to query unstructured data and use it to train other models, as well as the use of MongoDB Change Streams and Queryable Encryption for event processing and privacy. Overall, both Crewmate and Ada value MongoDB's flexible data storage, integration with AI models, and support for their AI-powered applications. They appreciate the performance, support, and independence from a single cloud vendor that MongoDB Atlas offers. Read: https://www.mongodb.com/blog/post/mongodb-helps-sewc-build-next-generation-manufacturing-execution-system-cn

Architecting Hyper-Scalable Infrastructure for AI and ML-Driven Fintech with Oracle’s Globally Distributed Database Empowering a US based fintech achieve hyper scalability in their architecture with Oracle’s Globally Distributed Database Read: https://blogs.oracle.com/database/post/architecting-hyperscalable-infrastructure-for-ai-and-mldriven-fintech-with-oracles-globally-distributed-database

Building AI With MongoDB: Optimizing the Product Lifecycle with Real-Time Customer Data Read: https://www.mongodb.com/blog/post/building-ai-mongodb-optimizing-product-lifecycle-real-time-customer-data

Бесконечные проверки – к успешному развитию: как мы обеспечиваем качество данных Привет, Хабр! Меня зовут Яна и я работаю Data Quality в департаменте развития аналитики "Цепочки поставок и поддерживающие функции" X5 Tech. В этой статье мы с моей коллегой Наташей, менеджером по качеству данных, решили рассказать о мониторинге качества данных большинства отчётов нашей команды. На первый взгляд может показаться, что проверять таблицы – задача рутинная и однотипная, но это не так, ведь все данные имеют свои особенности, а значит и проверки для них зачастую создаются уникальные. Статья, как нам кажется, будет полезна тем, кто интересуется качеством данных, ищет подходы к мониторингу или хочет больше узнать о работе DQ в целом. Читать: https://habr.com/ru/companies/X5Tech/articles/779856/

«Есть глюоны, кварки, виртуальные фотоны и… ячейки памяти»: что такое LUN-СХД, или как мы виртуализировали хранилище Вся жизнь — это выбор. Между Apple и Android, MySQL и PostgreSQL, здоровым питанием и тортом после 18:00. Но как быть, если его хочется, а отрабатывать калории в зале — не очень? Обычно весь торт есть не заставляют, поэтому его можно нарезать небольшими частями и использовать по необходимости. С данными в СХД такая же ситуация: можно использовать все пространство, а можно ограничиться LUN и оптимизировать бюджет. О том, что такое LUN, когда и где лучше использовать технологию — читайте в материале. Читать: https://habr.com/ru/companies/selectel/articles/779826/