
Contact.
Email. [email protected]
Phone. 010-7360-8122
Channel.
GitHub. https://github.com/SteveArseneLee
Blog. https://squidengineer.tistory.com/
πββοΈ Careers
- Okestro / Team. Data Solution / Data Engineer 2023.09.11 ~
- (Currently) Design and implement IoT data collection pipelines
- Spring boot, Kafka, Nifi, ActiveMQ
- Build and Operate Data Infrastructure for LLM Solution
- Kubernetes, Kafka, Harbor
- Cloud System Data Collection for Research
- Kubernetes, Prometheus, Loki, Grafana, Istio
π¨π»βπ» Projects
- IoT Data Collection Pipeline Design and Implementation
- 2024.06 ~ / Okestro
- Purpose : Design and build a data pipeline to reliably collect, integrate, and transmit data from various IoT devices
- Key Contributions
- Optimized data processing flow through Kafka topic separation by data source and message transfer structure design
- Implemented refinement logic to unify source-specific data formats such as temperature data (Celsius, Fahrenheit, etc.)
- Processed more than 1,000 data transactions per second and maintained a data loss rate of less than 10 percent
- Establish an operating environment that can process IoT data in real time and lay the foundation for utilization
- Tech stack used : Spring Boot, ActiveMQ, Kafka, Nifi
- Build and Operate Data Infrastructure for LLM Solution
- 2024.04 ~ 2024.06 / Okestro
- Purpose : Design data infrastructure for LLM solution and secure operational efficiency
- Key Contributions
- Newly built infrastructure by converting monolithic architecture to MSA
- Tested data transfer to ensure the answer order of the LLM engine
- Built a container image management system utilizing Harbor to improve container deployment and management efficiency
- Tech stack used : Kubernetes, Kafka, Harbor, Prometheus, Loki
- Cloud System Data Collection for Research
- 2023.09 ~ 2024.03 / Okestro
- Purpose : Collect metrics, logs, and distributed trace data for research
- Key Contributions
- Performed 300 load tests (stress-ng) on 4 metrics across 14 services and analyzed performance
- Designed and executed API load tests utilizing Locust based on 5 scenarios
- Built monitoring environment and collected data utilizing Prometheus, Loki, Grafana, etc.
- Share data collection and analysis know-how through in-house seminars
- Tech stack used : Kubernetes, Prometheus, Loki, Grafana, Jaeger, Istio
- Integrated Investment Service(github.com/SteveArseneLee/IIS/branches)
- 2022.09 ~ 2022.12 / KyungHee Univ.
- Purpose : Real-time integration of distributed data by leveraging Data Federation
- Key Contributions
- Designed data streaming pipeline based on Kafka and implemented data processing using Spark
- Automated data workflows with Airflow to secure a stable data processing environment
- Utilized AWS EC2, S3, and GCP GCS to build and efficiently manage data storage
- Document project results to demonstrate academic contributions and practical applicability
- Tech stack used : Kafka, Spark, Airflow, AWS, GCP, Snowflake
π Education and training
- SSAFY 10th (2023.07.05 ~ 2023.08.31)
- Kyung-Hee Univ. Department of Computer Science of Engineering. (2021.03 ~ 2023.08 / Bachelorβs degree)
- Dankook Univ. Department of Software Science (2016.03 ~ 2021.02 / Bachelorβs degree)
π
Qualifications
- OPIc IH (2023.04.26)
- Engineer Information Processing (2023.09.01)