Posts

Yarn

• ResourceManager, NodeManager, ApplicationMaster, Container - 22 December 2016
• Apache Hadoop YARN 핵심 소개 - 22 December 2016
• YARN 아키텍처_ResourceManager - 28 November 2016
• YARN 아키텍처_NodeManager - 28 November 2016

Spark

• Shuffle에 대해 알아보기 - 17 March 2019
• RDD가 Immutable한 이유 - 16 March 2019
• Spark Join에 대한 정리 - 23 December 2018
• Mastering Spark in Scalable Algorithms - 02 July 2018
• 스파크 최적화 및 디버깅 - 13 April 2017
• 고급 스파크 프로그래밍 - 12 April 2017
• 키/페어로 작업하기 - 12 April 2017
• 스파크 맛보기 - 12 April 2017

Kafka

• Exactly Once Semantics in Apache Kafka - 08 July 2019
• Optimizing Kafka Deployment Paper 정리 - 28 April 2019
• Kafka consumer Offset 체크 bash - 01 April 2019
• Kafka 브로커 재시작 프로세스 및 변화 - 08 March 2019
• Stream Processing In Kafka - 11 April 2018
• Building Data Pipeline In Kafka - 21 March 2018
• Kafka Internal - 02 March 2018
• Reliable Data Delivery In Kafka - 28 February 2018
• Kafka Consumer - 21 February 2018
• Kafka Produer - 07 February 2018
• Meet Kafka - 31 January 2018

Spark-streaming

• Spark Streaming With Kafka Direct API - 30 May 2018
• Spark Streaming Configuration - 30 May 2018

Gc

• Garbage Collection Monitoring - 02 April 2019
• Garbage Collection - 02 October 2018

High-performance-spark

• [HP-Spark] 파티셔너와 키/값 데이터 - 14 June 2019
• [HP-Spark] groupByKey 함수는 왜 그렇게 위험한가? - 14 June 2019
• [HP-Spark] 키/값 데이터로 작업하기 - 13 May 2019
• [HP-Spark] 효율적인 트랜스포메이션 : Part 3 - 14 April 2019
• [HP-Spark] 효율적인 트랜스포메이션 : Part 2 - 10 April 2019
• [HP-Spark] 효율적인 트랜스포메이션 : Part 1 - 07 April 2019
• [HP-Spark] DataFrame/DataSet와 Catalyst Optimizer - 22 March 2019
• [HP-Spark] 내가 생각하는 스파크 - 16 March 2019
• [HP-Spark] 스파크 잡에 대해 알아보기 - 12 March 2019
• [HP-Spark] 스파크 병렬 연산 모델 : RDD Part 2 - 10 February 2019
• [HP-Spark] 스파크 병렬 연산 모델 : RDD Part 1 - 08 February 2019

Hadoop

• WebHDFS REST API를 통한 Active NameNode 확인 - 12 February 2019

Spark-structured-streaming

• [Streaming] Spark structured Streaming : OutputMode - 15 February 2019
• [Streaming] Spark structured Streaming : 기본편 - 14 February 2019

Nifi

• [NiFi] DistributedMapCache 프로세서 - 10 July 2019
• [NiFi] NiFi Custom Processor 만들기 - 04 April 2019
• [NiFi] NiFi 권장 설정값 - 29 March 2019
• [NiFi] NiFi을 이용한 Database 연동 - 06 March 2019
• [NiFi] Throughtput 향상을 위한 작업들 - 27 February 2019
• [NiFi] NiFi Concept #2 - 25 February 2019
• [NiFi] NiFi Concept #1 - 25 February 2019

Etc

• [ETC] 한글 인코딩 euc-kr 과 UTF-8 - 11 July 2019
• [ETC] 정규식 특정 문자 제외한 결과 받아내기 - 26 June 2019
• [ETC] GitLab CI/CD 설정 - 16 April 2019
• [ETC] curl 인증서와 함께 https rest-api 실행 - 28 February 2019

Elasticsearch

• [ELASTIC SEARCH] Too many open file Error & Index read-only problem - 18 April 2019
• [ELASTIC SEARCH] bulk api 사용시 고유id값 설정 - 05 March 2019

Machine_learning

• [ML] LDA Topic Modeling - 19 March 2019

Spark-paper

• [Spark-Paper] Spark SQL: Relational Data Processing in Spark - 28 June 2019
• [Spark-Paper] RDD : A Fault-Tolerant Abstraction for In-Memory Cluster Computing - 19 April 2019

Nosql

• [NoSQL] Ch6. 버전 스탬프 - 12 May 2019
• [NoSQL] Ch5. 일관성 - 11 May 2019
• [NoSQL] Ch4. 분산 모델 - 06 May 2019
• [NoSQL] Ch3. 데이터 모델 상세 - 06 May 2019
• [NoSQL] Ch2. 집합적 데이터 모델 - 06 May 2019
• [NoSQL] Ch1. 왜 NoSQL 인가? - 05 May 2019

Data-intensive-application

• [Data-Intensive] Ch7. 트랜잭션 (Transaction) - 14 July 2019
• [Data-Intensive] Ch6. 파티셔닝 (Partitioning) - 17 June 2019
• [Data-Intensive] Ch5. 복제 (Leaderless Replication) - 08 June 2019
• [Data-Intensive] Ch5. 복제 (Multi-leader based Replication) - 07 June 2019
• [Data-Intensive] Ch5. 복제 (Leader-based Replication) - 01 June 2019
• [Data-Intensive] Ch_intro. 분산 데이터 - 30 May 2019
• [Data-Intensive] Ch4. 부호화와 발전 - 25 May 2019
• [Data-Intensive] Ch3. 저장소와 검색 - OLTP/OLAP/DW - 22 May 2019
• [Data-Intensive] Ch3. 저장소와 검색 - 19 May 2019
• [Data-Intensive] Ch1. 신뢰성,확장성,유지보수성 - 18 May 2019

Java

• Lock & ReentrantLock - 27 July 2019
• BlockingQueue 구현체 정리 - 26 July 2019
• BlockingQueue 인터페이스 정리 - 25 July 2019
• BlockingQueue VS ConcurrentLinkedQueue - 22 July 2019

Flink

• Flink 1.10.0 Taskmanager Memory Concepts & Configuration - 16 February 2020

Ignite

• [Ignite] Performace and Trobleshooting Guide - 20 June 2020
• [Ignite] Configurations - 20 June 2020
• [Ignite] Durable Memory Architecture - 14 June 2020
• [Ignite] Architecture deep dive Intro - 14 June 2020