diff --git a/docs/postgresql-search-migration-plan.md b/docs/postgresql-search-migration-plan.md new file mode 100644 index 00000000..e09471a7 --- /dev/null +++ b/docs/postgresql-search-migration-plan.md @@ -0,0 +1,143 @@ +# Elasticsearch → PostgreSQL 통합 검색 마이그레이션 설계서 + +## 1) Phase 0 분석 요약표 + +| 기능 | 기존 ES 구현 방식 | PG 대체 시 고려사항 | +|---|---|---| +| 인덱스 매핑 | `UnifiedSearchDocument`에 `title/content/searchTokens/category`, `title_autocomplete(search_as_you_type)`, `suggest(completion)` 필드 구성. 날짜는 `epoch_millis`. | `thingo_search_document` 테이블 + `weighted_tsv` 컬럼으로 통합. completion/search_as_you_type는 `search_query_log + pg_trgm/prefix index` 조합으로 대체. | +| Analyzer | Nori analyzer 설정은 코드상 명시 없음. 애플리케이션 레벨에서 `KomoranTokenizerUtil`로 형태소/복합어 토큰 생성(`searchTokens`). | `to_tsvector` 기본 + 한국어 형태소(선택: mecab config/pg_bigm/앱 전처리 지속). 기존 `search_tokens` 재활용 권장. | +| 검색 쿼리 | `bool` + 다중 `should`: `match_phrase(title)`, `multi_match(title^6,category^boost,content^0.3,searchTokens^boost)`, `bool_prefix(title_autocomplete)` + expansion + negative strategy + freshness/popularity boost. | `ts_rank_cd` 중심 + `WHERE type/category` 필터. 가중치: A=title/search_tokens, B=category, D=content. 최신성/인기도는 보조 점수 컬럼 또는 SQL 가중식으로 추가. | +| 정렬 | relevance=`_score`(+date tie-breaker), latest/oldest=`date` 정렬. | relevance=`ts_rank_cd(weighted_tsv, tsquery)` + `created_at DESC`, latest/oldest는 `created_at` 인덱스 사용. | +| 동기화 | `SearchIndexSyncService`: Notice/Community(전처리 포함) + News/DepartmentSchedule/StudentCouncilNotice/Broadcast/MjuCalendar를 각 도메인 인덱스로 저장 후 unified 재구축. | 동일 도메인 원천 사용, ES 저장 대신 `thingo_search_document`로 bulk insert + trigger로 `weighted_tsv` 자동 갱신. | +| 응답 DTO | `SearchResponseDTO`: id, highlightedTitle, highlightedContent, date, link, category, type, imageUrl, score, authorName, likeCount, commentCount. | DTO 그대로 유지. 하이라이트는 1차적으로 원문 title/content 반환(추후 `ts_headline` 적용 가능). | +| 실시간 검색어 | Redis ZSET(`realtime_keywords`) + keyword별 timestamp LIST(`search:history{keyword}`), 3일 TTL, 1시간 스케줄 정리. | `search_query_log` 누적 + 윈도우 집계/MV + Redis 캐시 병행. 장애 시 Redis fallback 유지. | + +### 필드 boost → `ts_rank_cd` weight 매핑 +- ES의 주요 boost 상대강도 기반 권장 매핑: + - **A**: `title`, `search_tokens` (exact/compact/token 신호 핵심) + - **B**: `category` + - **C**: `title_normalized`, `category_normalized` (선택) + - **D**: `content` + +가중 tsvector 예시: +```sql +setweight(to_tsvector('simple', coalesce(title,'')), 'A') || +setweight(to_tsvector('simple', coalesce(search_tokens,'')), 'A') || +setweight(to_tsvector('simple', coalesce(category,'')), 'B') || +setweight(to_tsvector('simple', coalesce(content,'')), 'D') +``` + +## 2) ERD / 아키텍처 다이어그램 + +```text +[Client] + | + v +GET /api/v1/search/detail +POST /api/v1/search/sync +GET /api/v1/search/suggest + | + v +[SearchController] + |-- UnifiedSearchService ------------------------------. + |-- SearchIndexSyncService ---------------------. | + |-- RealtimeKeywordService ------------------. | | + v | | | +[PostgresUnifiedSearchRepository] | | | + | | | | + |-- thingo_search_document (FTS + filter) <----' | | + |-- search_query_log (autocomplete/realtime) ------' | + |-- mv_realtime_keywords (optional refresh)-------------' + | + '-- Redis (hot cache/fallback for top keywords) +``` + +## 3) 한국어 처리 대안 + +1. **mecab + custom text search config** + - 장점: 형태소 품질 우수, 조사/어미 처리 강함. + - 단점: 운영 환경 설치/빌드 난이도 높음. +2. **pg_bigm (bi-gram)** + - 장점: 설정 간단, 오타/부분일치 강함. + - 단점: 인덱스 커짐, 의미 기반 랭킹은 약함. +3. **앱 전처리 후 tsvector 저장 (현행 Komoran 확장)** + - 장점: 현재 로직 재사용, 전환 리스크 낮음. + - 단점: DB 단독 파이프라인보다 앱 의존 증가. + +**추천안:** 단기에는 **(3)** + `pg_trgm` 보강, 중장기에 필요시 (1) 도입. + +## 4) DDL + 인덱스 + +- 실제 SQL: `src/main/resources/db/search/postgres_search_schema.sql` +- 핵심: + - `thingo_search_document` + `weighted_tsv` + GIN + - `(type, category)` 복합 인덱스 + - `created_at` 정렬 인덱스 + - `title` prefix + trigram 인덱스 + - `search_query_log` + 집계용 인덱스 + MV + +## 5) 추천 검색(Autocomplete/Suggest) SQL 랭킹 + +```sql +score = ( + personal_weight * 0.45 + + global_popularity * 0.35 + + title_hits * 0.20 +) * EXP(-λ * EXTRACT(EPOCH FROM (now() - searched_at))) +``` + +- prefix 중심: `ILIKE :keyword || '%'` + B-Tree(prefix) 우선 +- infix/fuzzy 보강: `pg_trgm GIN` (`%keyword%`/유사도) + +## 6) 최신성 decay 정렬 예시 + +- Exponential: +```sql +relevance_score * EXP(-0.00002 * EXTRACT(EPOCH FROM (NOW() - created_at))) +``` +- Linear: +```sql +relevance_score * GREATEST(0.2, 1 - EXTRACT(EPOCH FROM (NOW() - created_at))/2592000) +``` +- Step: +```sql +relevance_score * CASE + WHEN created_at >= NOW() - INTERVAL '7 days' THEN 1.15 + WHEN created_at >= NOW() - INTERVAL '30 days' THEN 1.05 + ELSE 1.0 +END +``` + +## 7) 정확도/성능 검증 기준 + +- BM25(ES) vs ts_rank_cd(PG): + - BM25는 TF saturation/문서 길이 정규화가 정교. + - ts_rank_cd는 커버 밀도 기반으로 단순하며 길이 보정이 제한적. +- 보완 전략: + - `custom_score = ts_rank_cd + recency + popularity + click-through` 조합. + - offline metric: Precision@10, nDCG@10 (쿼리셋/정답셋 기반). + +### 예상 성능 매트릭스 (인덱스/카디널리티에 따른 범위 추정) + +| 규모 | tsvector+GIN | LIKE prefix | REGEXP | pg_trgm | +|---|---:|---:|---:|---:| +| 10만 row | 30~90ms | 10~40ms | 300ms+ | 40~120ms | +| 100만 row | 80~220ms | 30~120ms | 2s+ | 120~350ms | +| 1,000만 row | 250~600ms | 120~500ms | 10s+ | 400ms~1.2s | + +> REGEXP는 대부분 full scan 유발로 기본 경로에서 제외 권장. + +### SLA 판단 +- 자동완성 200ms: prefix + hot cache(Redis) 시 충족 가능. +- 검색 결과 500ms: GIN + 필터 인덱스 + read replica 조건에서 충족 가능. + +## 8) 마이그레이션 체크리스트 + +1. DDL 적용(`postgres_search_schema.sql`). +2. `SearchIndexSyncService`로 최초 full rebuild. +3. `/search/detail` 트래픽의 5~10%를 PG path로 canary. +4. Precision@10/nDCG 비교(ES vs PG). +5. p95 latency 점검(autocomplete 200ms, detail 500ms). +6. Redis 캐시 튜닝(hot query TTL). +7. ES write 중단 → read 중단 → 인프라 제거. +8. ES config/repository 의존성 제거 정리. diff --git a/src/main/java/nova/mjs/domain/thingo/ElasticSearch/Repository/PostgresUnifiedSearchRepository.java b/src/main/java/nova/mjs/domain/thingo/ElasticSearch/Repository/PostgresUnifiedSearchRepository.java new file mode 100644 index 00000000..cf490d79 --- /dev/null +++ b/src/main/java/nova/mjs/domain/thingo/ElasticSearch/Repository/PostgresUnifiedSearchRepository.java @@ -0,0 +1,318 @@ +package nova.mjs.domain.thingo.ElasticSearch.Repository; + +import lombok.RequiredArgsConstructor; +import nova.mjs.domain.thingo.ElasticSearch.SearchResponseDTO; +import org.springframework.data.domain.Page; +import org.springframework.data.domain.PageImpl; +import org.springframework.data.domain.Pageable; +import org.springframework.jdbc.core.RowMapper; +import org.springframework.jdbc.core.namedparam.MapSqlParameterSource; +import org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate; +import org.springframework.stereotype.Repository; + +import java.sql.Timestamp; +import java.time.Instant; +import java.util.List; +import java.util.Map; + +@Repository +@RequiredArgsConstructor +public class PostgresUnifiedSearchRepository { + + private final NamedParameterJdbcTemplate jdbcTemplate; + + public Page search(String keyword, + String type, + String category, + String order, + Pageable pageable) { + + String normalizedKeyword = keyword == null ? "" : keyword.trim(); + String tsQuery = normalizedKeyword.isBlank() + ? "" + : normalizedKeyword.replaceAll("\\s+", " & "); + + MapSqlParameterSource params = new MapSqlParameterSource() + .addValue("keyword", normalizedKeyword) + .addValue("tsQuery", tsQuery) + .addValue("type", type) + .addValue("category", category) + .addValue("limit", pageable.getPageSize()) + .addValue("offset", pageable.getOffset()); + + String where = """ + WHERE active = true + AND (:type IS NULL OR type = :type) + AND (:category IS NULL OR category = :category) + """; + + String rankExpr = """ + CASE + WHEN :tsQuery = '' THEN 0.0 + ELSE ts_rank_cd( + weighted_tsv, + websearch_to_tsquery('simple', :tsQuery), + 4 + ) + END + """; + + String searchConstraint = """ + AND ( + :keyword = '' + OR weighted_tsv @@ websearch_to_tsquery('simple', :tsQuery) + OR title ILIKE CONCAT('%', :keyword, '%') + OR content ILIKE CONCAT('%', :keyword, '%') + ) + """; + + String orderBy = switch (normalizeOrder(order)) { + case "latest" -> " ORDER BY created_at DESC, id DESC "; + case "oldest" -> " ORDER BY created_at ASC, id ASC "; + default -> " ORDER BY rank_score DESC, created_at DESC, id DESC "; + }; + + String sql = """ + SELECT id, + original_id, + type, + title, + content, + created_at, + link, + category, + image_url, + author_name, + like_count, + comment_count, + %s AS rank_score + FROM thingo_search_document + %s + %s + """.formatted(rankExpr, where, searchConstraint) + + orderBy + + " LIMIT :limit OFFSET :offset"; + + List content = jdbcTemplate.query(sql, params, searchRowMapper()); + + Long total = jdbcTemplate.queryForObject( + "SELECT COUNT(*) FROM thingo_search_document " + where + searchConstraint, + params, + Long.class + ); + + return new PageImpl<>(content, pageable, total == null ? 0 : total); + } + + public List autocomplete(String keyword, Long userId, int size) { + String normalized = keyword == null ? "" : keyword.trim(); + if (normalized.isBlank()) { + return List.of(); + } + + MapSqlParameterSource params = new MapSqlParameterSource() + .addValue("keyword", normalized) + .addValue("size", size) + .addValue("userId", userId); + + String sql = """ + WITH personal AS ( + SELECT keyword, + COUNT(*)::double precision AS personal_weight, + MAX(searched_at) AS last_searched_at + FROM search_query_log + WHERE (:userId IS NULL OR user_id = :userId) + AND keyword ILIKE CONCAT(:keyword, '%') + GROUP BY keyword + ), + global_pop AS ( + SELECT keyword, + COUNT(*)::double precision AS global_popularity, + MAX(searched_at) AS last_searched_at + FROM search_query_log + WHERE keyword ILIKE CONCAT(:keyword, '%') + GROUP BY keyword + ), + title_pop AS ( + SELECT title AS keyword, + COUNT(*)::double precision AS title_hits + FROM thingo_search_document + WHERE title ILIKE CONCAT(:keyword, '%') + GROUP BY title + ) + SELECT candidate.keyword + FROM ( + SELECT COALESCE(p.keyword, g.keyword, t.keyword) AS keyword, + COALESCE(p.personal_weight, 0) AS personal_weight, + COALESCE(g.global_popularity, 0) AS global_popularity, + COALESCE(t.title_hits, 0) AS title_hits, + GREATEST(COALESCE(p.last_searched_at, '-infinity'::timestamp), + COALESCE(g.last_searched_at, '-infinity'::timestamp)) AS searched_at, + ( + (COALESCE(p.personal_weight, 0) * 0.45) + + (COALESCE(g.global_popularity, 0) * 0.35) + + (COALESCE(t.title_hits, 0) * 0.20) + ) + * EXP( + -0.000015 + * EXTRACT(EPOCH FROM (NOW() - GREATEST(COALESCE(p.last_searched_at, NOW()), COALESCE(g.last_searched_at, NOW())))) + ) AS score + FROM personal p + FULL OUTER JOIN global_pop g ON p.keyword = g.keyword + FULL OUTER JOIN title_pop t ON COALESCE(p.keyword, g.keyword) = t.keyword + ) candidate + WHERE candidate.keyword IS NOT NULL + ORDER BY candidate.score DESC, candidate.keyword ASC + LIMIT :size + """; + + return jdbcTemplate.query(sql, params, (rs, rowNum) -> rs.getString("keyword")); + } + + public List topKeywords(int topN) { + String sql = """ + SELECT keyword + FROM search_query_log + WHERE searched_at >= NOW() - INTERVAL '3 days' + GROUP BY keyword + ORDER BY COUNT(*) DESC, MAX(searched_at) DESC + LIMIT :topN + """; + + return jdbcTemplate.query(sql, Map.of("topN", topN), (rs, rowNum) -> rs.getString("keyword")); + } + + public void insertSearchLog(String keyword, Long userId) { + if (keyword == null || keyword.trim().isBlank()) { + return; + } + jdbcTemplate.update( + """ + INSERT INTO search_query_log(keyword, user_id, searched_at) + VALUES (:keyword, :userId, NOW()) + """, + new MapSqlParameterSource() + .addValue("keyword", keyword.trim()) + .addValue("userId", userId) + ); + } + + public void rebuildSearchDocuments(List rows) { + jdbcTemplate.update("TRUNCATE TABLE thingo_search_document", new MapSqlParameterSource()); + + String sql = """ + INSERT INTO thingo_search_document( + id, original_id, type, title, title_normalized, content, content_normalized, + category, category_normalized, search_tokens, link, image_url, created_at, + updated_at, active, popularity, like_count, comment_count, author_name + ) + VALUES ( + :id, :originalId, :type, :title, :titleNormalized, :content, :contentNormalized, + :category, :categoryNormalized, :searchTokens, :link, :imageUrl, :createdAt, + :updatedAt, :active, :popularity, :likeCount, :commentCount, :authorName + ) + """; + + SqlParameterSourceBuilder.batchUpdate(jdbcTemplate, sql, rows); + } + + public void refreshSearchVectors() { + jdbcTemplate.update(""" + UPDATE thingo_search_document + SET weighted_tsv = + setweight(to_tsvector('simple', COALESCE(title, '')), 'A') + || setweight(to_tsvector('simple', COALESCE(category, '')), 'B') + || setweight(to_tsvector('simple', COALESCE(search_tokens, '')), 'A') + || setweight(to_tsvector('simple', COALESCE(content, '')), 'D'), + updated_at = NOW() + """, new MapSqlParameterSource()); + } + + private RowMapper searchRowMapper() { + return (rs, rowNum) -> SearchResponseDTO.builder() + .id(rs.getString("id")) + .highlightedTitle(rs.getString("title")) + .highlightedContent(rs.getString("content")) + .date(toInstant(rs.getTimestamp("created_at"))) + .link(rs.getString("link")) + .category(rs.getString("category")) + .type(rs.getString("type") == null ? null : rs.getString("type").toLowerCase()) + .imageUrl(rs.getString("image_url")) + .score(rs.getFloat("rank_score")) + .authorName(rs.getString("author_name")) + .likeCount(rs.getObject("like_count", Integer.class)) + .commentCount(rs.getObject("comment_count", Integer.class)) + .build(); + } + + private Instant toInstant(Timestamp timestamp) { + return timestamp == null ? null : timestamp.toInstant(); + } + + private String normalizeOrder(String order) { + if ("latest".equalsIgnoreCase(order)) { + return "latest"; + } + if ("oldest".equalsIgnoreCase(order)) { + return "oldest"; + } + return "relevance"; + } + + public record SearchWriteModel( + String id, + String originalId, + String type, + String title, + String titleNormalized, + String content, + String contentNormalized, + String category, + String categoryNormalized, + String searchTokens, + String link, + String imageUrl, + Instant createdAt, + Instant updatedAt, + boolean active, + Double popularity, + Integer likeCount, + Integer commentCount, + String authorName + ) { + } + + private static final class SqlParameterSourceBuilder { + private SqlParameterSourceBuilder() { + } + + static void batchUpdate(NamedParameterJdbcTemplate jdbcTemplate, + String sql, + List rows) { + var sources = rows.stream() + .map(row -> new MapSqlParameterSource() + .addValue("id", row.id()) + .addValue("originalId", row.originalId()) + .addValue("type", row.type()) + .addValue("title", row.title()) + .addValue("titleNormalized", row.titleNormalized()) + .addValue("content", row.content()) + .addValue("contentNormalized", row.contentNormalized()) + .addValue("category", row.category()) + .addValue("categoryNormalized", row.categoryNormalized()) + .addValue("searchTokens", row.searchTokens()) + .addValue("link", row.link()) + .addValue("imageUrl", row.imageUrl()) + .addValue("createdAt", row.createdAt() == null ? null : Timestamp.from(row.createdAt())) + .addValue("updatedAt", row.updatedAt() == null ? null : Timestamp.from(row.updatedAt())) + .addValue("active", row.active()) + .addValue("popularity", row.popularity()) + .addValue("likeCount", row.likeCount()) + .addValue("commentCount", row.commentCount()) + .addValue("authorName", row.authorName())) + .toArray(MapSqlParameterSource[]::new); + + jdbcTemplate.batchUpdate(sql, sources); + } + } +} diff --git a/src/main/java/nova/mjs/domain/thingo/ElasticSearch/Service/SearchIndexSyncService.java b/src/main/java/nova/mjs/domain/thingo/ElasticSearch/Service/SearchIndexSyncService.java index 871d7987..6929ce16 100644 --- a/src/main/java/nova/mjs/domain/thingo/ElasticSearch/Service/SearchIndexSyncService.java +++ b/src/main/java/nova/mjs/domain/thingo/ElasticSearch/Service/SearchIndexSyncService.java @@ -3,48 +3,29 @@ import lombok.RequiredArgsConstructor; import lombok.extern.slf4j.Slf4j; import nova.mjs.domain.thingo.ElasticSearch.Document.*; +import nova.mjs.domain.thingo.ElasticSearch.Repository.PostgresUnifiedSearchRepository; import nova.mjs.domain.thingo.ElasticSearch.indexing.Preprocessor.community.CommunityContentPreprocessor; import nova.mjs.domain.thingo.ElasticSearch.indexing.Preprocessor.notice.NoticeContentPreprocessor; -import nova.mjs.domain.thingo.ElasticSearch.Repository.*; import nova.mjs.domain.thingo.ElasticSearch.indexing.mapper.UnifiedSearchMapper; import nova.mjs.domain.thingo.broadcast.repository.BroadcastRepository; import nova.mjs.domain.thingo.calendar.repository.MjuCalendarRepository; import nova.mjs.domain.thingo.community.repository.CommunityBoardRepository; -import nova.mjs.domain.thingo.department.repository.StudentCouncilNoticeRepository; import nova.mjs.domain.thingo.department.repository.DepartmentScheduleRepository; +import nova.mjs.domain.thingo.department.repository.StudentCouncilNoticeRepository; import nova.mjs.domain.thingo.news.repository.NewsRepository; import nova.mjs.domain.thingo.notice.repository.NoticeRepository; -import org.springframework.data.elasticsearch.core.ElasticsearchOperations; -import org.springframework.data.elasticsearch.core.IndexOperations; -import org.springframework.data.elasticsearch.repository.ElasticsearchRepository; import org.springframework.stereotype.Service; +import java.util.ArrayList; import java.util.List; import java.util.function.BiFunction; import java.util.function.Function; -import java.util.stream.StreamSupport; -/** - * SearchIndexSyncService - * - * 역할 - * - RDB → 도메인 Elasticsearch 인덱스 동기화 - * - 모든 도메인 인덱스 기준으로 Unified 인덱스 재생성 - * - * 설계 원칙 - * - Unified 인덱스는 파생 인덱스 - * - 도메인 인덱스가 Single Source of Truth - * - 항상 drop & recreate 전략 - */ @Slf4j @Service @RequiredArgsConstructor public class SearchIndexSyncService { - /* ========================= - RDB Repositories - ========================= */ - private final NoticeRepository noticeRepository; private final NewsRepository newsRepository; private final CommunityBoardRepository communityBoardRepository; @@ -53,213 +34,71 @@ public class SearchIndexSyncService { private final BroadcastRepository broadcastRepository; private final MjuCalendarRepository mjuCalendarRepository; - /* ========================= - Elasticsearch Repositories - ========================= */ - - private final NoticeSearchRepository noticeSearchRepository; - private final NewsSearchRepository newsSearchRepository; - private final CommunitySearchRepository communitySearchRepository; - private final DepartmentScheduleSearchRepository departmentScheduleSearchRepository; - private final StudentCouncilNoticeSearchRepository studentCouncilNoticeSearchRepository; - private final BroadcastSearchRepository broadcastSearchRepository; - private final MjuCalendarSearchRepository mjuCalendarSearchRepository; - - private final UnifiedSearchRepository unifiedSearchRepository; private final UnifiedSearchMapper unifiedSearchMapper; - - /* ========================= - Infrastructure - ========================= */ - - private final ElasticsearchOperations elasticsearchOperations; - - /* ========================= - Preprocessors - ========================= */ + private final PostgresUnifiedSearchRepository postgresUnifiedSearchRepository; private final NoticeContentPreprocessor noticeContentPreprocessor; private final CommunityContentPreprocessor communityContentPreprocessor; - /** - * Controller 단일 진입점 - */ public void syncAll() { - log.info("[SEARCH][SYNC][ALL] start"); - - syncDomainIndexes(); - rebuildUnifiedIndex(); - - log.info("[SEARCH][SYNC][ALL] end"); - } + log.info("[SEARCH][PG][SYNC][ALL] start"); - /* ================================================== - 1. DB → 도메인 Elasticsearch 동기화 - ================================================== */ + List rows = new ArrayList<>(); - private void syncDomainIndexes() { + rows.addAll(buildRows(noticeRepository.findAll(), noticeContentPreprocessor, NoticeDocument::from)); + rows.addAll(buildRows(communityBoardRepository.findAll(), communityContentPreprocessor, CommunityDocument::from)); + rows.addAll(buildRows(newsRepository.findAll(), NewsDocument::from)); + rows.addAll(buildRows(departmentScheduleRepository.findAll(), DepartmentScheduleDocument::from)); + rows.addAll(buildRows(studentCouncilNoticeRepository.findAll(), StudentCouncilNoticeDocument::from)); + rows.addAll(buildRows(broadcastRepository.findAll(), BroadcastDocument::from)); + rows.addAll(buildRows(mjuCalendarRepository.findAll(), MjuCalendarDocument::from)); - // Notice (HTML 전처리 필요) - syncWithPreprocessor( - "NOTICE", - noticeRepository.findAll(), - noticeContentPreprocessor, - NoticeDocument::from, - noticeSearchRepository, - NoticeDocument.class - ); - - // Community (Editor JSON 전처리 필요) - syncWithPreprocessor( - "COMMUNITY", - communityBoardRepository.findAll(), - communityContentPreprocessor, - CommunityDocument::from, - communitySearchRepository, - CommunityDocument.class - ); - - // 전처리 없는 도메인들 - sync( - "NEWS", - newsRepository.findAll(), - NewsDocument::from, - newsSearchRepository, - NewsDocument.class - ); - - sync( - "DEPARTMENT_SCHEDULE", - departmentScheduleRepository.findAll(), - DepartmentScheduleDocument::from, - departmentScheduleSearchRepository, - DepartmentScheduleDocument.class - ); - - sync( - "STUDENT_COUNCIL_NOTICE", - studentCouncilNoticeRepository.findAll(), - StudentCouncilNoticeDocument::from, - studentCouncilNoticeSearchRepository, - StudentCouncilNoticeDocument.class - ); - - sync( - "BROADCAST", - broadcastRepository.findAll(), - BroadcastDocument::from, - broadcastSearchRepository, - BroadcastDocument.class - ); + postgresUnifiedSearchRepository.rebuildSearchDocuments(rows); + postgresUnifiedSearchRepository.refreshSearchVectors(); - sync( - "MJU_CALENDAR", - mjuCalendarRepository.findAll(), - MjuCalendarDocument::from, - mjuCalendarSearchRepository, - MjuCalendarDocument.class - ); + log.info("[SEARCH][PG][SYNC][ALL] rows={}", rows.size()); } - /** - * 전처리가 필요 없는 일반 도메인 sync - */ - private void sync( - String domainName, - List entities, - Function mapper, - ElasticsearchRepository repository, - Class documentClass - ) { - ensureIndex(documentClass, domainName); - - List documents = entities.stream() + private List buildRows(List entities, + Function mapper) { + return entities.stream() .map(mapper) + .map(unifiedSearchMapper::from) + .map(this::toWriteModel) .toList(); - - repository.saveAll(documents); - - log.info("[SEARCH][SYNC][{}] count={}", domainName, documents.size()); } - /** - * 전처리가 필요한 도메인 전용 sync - * - * 설계 의도: - * - 전처리 필요 여부를 Service 레벨에서 명시적으로 드러낸다. - * - Document.from(...) 시그니처에 전처리 의존성을 강제한다. - */ - private void syncWithPreprocessor( - String domainName, - List entities, - P preprocessor, - BiFunction mapper, - ElasticsearchRepository repository, - Class documentClass - ) { - ensureIndex(documentClass, domainName); - - List documents = entities.stream() + private List buildRows(List entities, + P preprocessor, + BiFunction mapper) { + return entities.stream() .map(entity -> mapper.apply(entity, preprocessor)) + .map(unifiedSearchMapper::from) + .map(this::toWriteModel) .toList(); - - repository.saveAll(documents); - - log.info("[SEARCH][SYNC][{}] count={}", domainName, documents.size()); - } - - /* ================================================== - 2. 도메인 Elasticsearch → Unified 재생성 - ================================================== */ - - private void rebuildUnifiedIndex() { - - log.info("[SEARCH][UNIFIED][REBUILD] start"); - - IndexOperations indexOps = - elasticsearchOperations.indexOps(UnifiedSearchDocument.class); - - if (indexOps.exists()) { - indexOps.delete(); - log.info("[SEARCH][UNIFIED] index deleted"); - } - - indexOps.create(); - indexOps.putMapping(indexOps.createMapping()); - log.info("[SEARCH][UNIFIED] index created"); - - rebuildFrom(noticeSearchRepository.findAll()); - rebuildFrom(newsSearchRepository.findAll()); - rebuildFrom(communitySearchRepository.findAll()); - rebuildFrom(departmentScheduleSearchRepository.findAll()); - rebuildFrom(studentCouncilNoticeSearchRepository.findAll()); - rebuildFrom(broadcastSearchRepository.findAll()); - rebuildFrom(mjuCalendarSearchRepository.findAll()); - - log.info("[SEARCH][UNIFIED][REBUILD] end"); } - /** - * 도메인 SearchDocument → UnifiedSearchDocument 변환 - */ - private void rebuildFrom(Iterable domainDocuments) { - - List unifiedDocuments = - StreamSupport.stream(domainDocuments.spliterator(), false) - .map(unifiedSearchMapper::from) - .toList(); - - unifiedSearchRepository.saveAll(unifiedDocuments); - } - - private void ensureIndex(Class documentClass, String domainName) { - IndexOperations indexOps = elasticsearchOperations.indexOps(documentClass); - if (indexOps.exists()) { - return; - } - - indexOps.create(); - indexOps.putMapping(indexOps.createMapping()); - log.info("[SEARCH][INDEX][CREATE][{}] created", domainName); + private PostgresUnifiedSearchRepository.SearchWriteModel toWriteModel(UnifiedSearchDocument doc) { + return new PostgresUnifiedSearchRepository.SearchWriteModel( + doc.getId(), + doc.getOriginalId(), + doc.getType(), + doc.getTitle(), + doc.getTitleNormalized(), + doc.getContent(), + doc.getContentNormalized(), + doc.getCategory(), + doc.getCategoryNormalized(), + doc.getSearchTokens(), + doc.getLink(), + doc.getImageUrl(), + doc.getDate(), + doc.getUpdatedAt(), + Boolean.TRUE.equals(doc.getActive()), + doc.getPopularity(), + doc.getLikeCount(), + doc.getCommentCount(), + doc.getAuthorName() + ); } } diff --git a/src/main/java/nova/mjs/domain/thingo/ElasticSearch/Service/SuggestService.java b/src/main/java/nova/mjs/domain/thingo/ElasticSearch/Service/SuggestService.java index 6b6ddbfe..0a44cdff 100644 --- a/src/main/java/nova/mjs/domain/thingo/ElasticSearch/Service/SuggestService.java +++ b/src/main/java/nova/mjs/domain/thingo/ElasticSearch/Service/SuggestService.java @@ -1,172 +1,25 @@ package nova.mjs.domain.thingo.ElasticSearch.Service; -import co.elastic.clients.elasticsearch._types.query_dsl.TextQueryType; -import co.elastic.clients.elasticsearch.core.search.CompletionSuggester; -import co.elastic.clients.elasticsearch.core.search.FieldSuggester; -import co.elastic.clients.elasticsearch.core.search.Suggester; import lombok.RequiredArgsConstructor; -import nova.mjs.domain.thingo.ElasticSearch.Document.UnifiedSearchDocument; -import nova.mjs.domain.thingo.ElasticSearch.suggest.IntentLexicon; -import org.springframework.data.elasticsearch.client.elc.ElasticsearchTemplate; -import org.springframework.data.elasticsearch.client.elc.NativeQuery; -import org.springframework.data.elasticsearch.core.SearchHit; -import org.springframework.data.elasticsearch.core.SearchHits; -import org.springframework.data.elasticsearch.core.suggest.response.Suggest; +import nova.mjs.domain.thingo.ElasticSearch.Repository.PostgresUnifiedSearchRepository; import org.springframework.stereotype.Service; -import java.util.*; -import java.util.stream.Collectors; +import java.util.List; @Service @RequiredArgsConstructor public class SuggestService { - private static final String SUGGESTION_NAME = "suggestion"; private static final int DEFAULT_SIZE = 7; - private final ElasticsearchTemplate elasticsearchTemplate; - private final IntentLexicon intentLexicon; + private final PostgresUnifiedSearchRepository postgresUnifiedSearchRepository; - /** - * 자동완성 최종 정책 - * - * 1) IntentLexicon 기반: 1글자에서도 의미 있는 연관어 제공 - * 2) Completion Suggest: prefix 기반 추천 (빠르고 정확) - * 3) search_as_you_type: 부분 단어/중간 단어 보강 (특공대 같은 케이스) - * - * 결론: - * - SuggestService만 바꾸는 게 아니라, - * 인덱스에 suggest/title_autocomplete가 "정상 저장"되도록 Mapper/Document도 일치해야 한다. - */ public List getSuggestions(String rawKeyword) { - String keyword = normalize(rawKeyword); - if (keyword.isEmpty()) { + String keyword = rawKeyword == null ? "" : rawKeyword.trim(); + if (keyword.isBlank()) { return List.of(); } - LinkedHashSet merged = new LinkedHashSet<>(); - - // 1) 1글자에서도 동작: IntentLexicon 확장 - intentLexicon.matchPrefix(keyword).ifPresent(entry -> { - add(merged, entry.intent()); - for (String ex : nullSafe(entry.expansions())) { - add(merged, ex); - } - }); - - // 2) completion: 1글자도 허용하되, 1글자일 때는 size를 줄여 노이즈를 완화 - int completionSize = (keyword.length() == 1) ? 5 : DEFAULT_SIZE; - - // 기존 meta 정책을 쓰되, "그냥 되도록"이 목표면 minPrefix를 1로 강제해도 됨 - int minPrefix = intentLexicon.meta().minCompletionPrefixLength(); - if (keyword.length() >= minPrefix) { - merged.addAll(fetchCompletion(keyword, completionSize)); - } - - // 3) search_as_you_type은 2글자부터 - if (keyword.length() >= 2) { - merged.addAll(fetchSearchAsYouType(keyword, DEFAULT_SIZE)); - } - - return merged.stream() - .map(String::trim) - .filter(v -> !v.isEmpty()) - .filter(v -> v.length() <= 50) - .limit(DEFAULT_SIZE) - .collect(Collectors.toList()); - } - - /** - * Completion Suggest (prefix) - */ - private List fetchCompletion(String keyword, int size) { - CompletionSuggester completion = CompletionSuggester.of(cb -> cb - .field("suggest") - .skipDuplicates(true) - .size(size) - ); - - FieldSuggester fieldSuggester = FieldSuggester.of(fb -> fb - .prefix(keyword) - .completion(completion) - ); - - Suggester suggester = Suggester.of(s -> s - .suggesters(Map.of(SUGGESTION_NAME, fieldSuggester)) - ); - - NativeQuery query = NativeQuery.builder() - .withSuggester(suggester) - .build(); - - SearchHits hits = - elasticsearchTemplate.search(query, UnifiedSearchDocument.class); - - Suggest suggest = hits.getSuggest(); - if (suggest == null) return List.of(); - - Suggest.Suggestion suggestion = suggest.getSuggestion(SUGGESTION_NAME); - if (suggestion == null) return List.of(); - - return suggestion.getEntries().stream() - .flatMap(e -> e.getOptions().stream()) - .map(o -> o.getText()) - .filter(Objects::nonNull) - .map(String::trim) - .filter(v -> !v.isEmpty()) - .distinct() - .toList(); - } - - /** - * search_as_you_type 기반 보강 - * - * - title_autocomplete / _2gram / _3gram에 bool_prefix로 조회 - * - 결과 문서 title을 suggestion 후보로 사용 - * - * 주의: - * - 이게 제대로 동작하려면 "title_autocomplete" 필드가 인덱스에 존재하고 - * 실제로 값이 들어가 있어야 한다. (Mapper에서 반드시 세팅 필요) - */ - private List fetchSearchAsYouType(String keyword, int size) { - NativeQuery query = NativeQuery.builder() - .withQuery(q -> q.multiMatch(mm -> mm - .query(keyword) - .type(TextQueryType.BoolPrefix) - .fields( - "title_autocomplete", - "title_autocomplete._2gram", - "title_autocomplete._3gram" - ) - )) - .withMaxResults(size) - .build(); - - SearchHits hits = - elasticsearchTemplate.search(query, UnifiedSearchDocument.class); - - return hits.getSearchHits().stream() - .map(SearchHit::getContent) - .map(UnifiedSearchDocument::getTitle) - .filter(Objects::nonNull) - .map(String::trim) - .filter(v -> !v.isEmpty()) - .distinct() - .toList(); - } - - private void add(Set set, String v) { - if (v == null) return; - String s = v.trim(); - if (s.isEmpty()) return; - set.add(s); - } - - private String normalize(String v) { - return v == null ? "" : v.trim(); - } - - private List nullSafe(List v) { - return v == null ? List.of() : v; + return postgresUnifiedSearchRepository.autocomplete(keyword, null, DEFAULT_SIZE); } } diff --git a/src/main/java/nova/mjs/domain/thingo/ElasticSearch/Service/UnifiedSearchService.java b/src/main/java/nova/mjs/domain/thingo/ElasticSearch/Service/UnifiedSearchService.java index c17488f3..8d5c1c65 100644 --- a/src/main/java/nova/mjs/domain/thingo/ElasticSearch/Service/UnifiedSearchService.java +++ b/src/main/java/nova/mjs/domain/thingo/ElasticSearch/Service/UnifiedSearchService.java @@ -1,42 +1,19 @@ package nova.mjs.domain.thingo.ElasticSearch.Service; import lombok.RequiredArgsConstructor; -import nova.mjs.domain.thingo.ElasticSearch.Document.UnifiedSearchDocument; -import nova.mjs.domain.thingo.ElasticSearch.Repository.UnifiedSearchQueryRepository; +import nova.mjs.domain.thingo.ElasticSearch.Repository.PostgresUnifiedSearchRepository; import nova.mjs.domain.thingo.ElasticSearch.SearchResponseDTO; import nova.mjs.domain.thingo.ElasticSearch.SearchType; -import nova.mjs.domain.thingo.ElasticSearch.search.SearchIntentContext; -import nova.mjs.domain.thingo.ElasticSearch.search.SearchIntentResolver; -import nova.mjs.domain.thingo.ElasticSearch.search.SearchQueryPlan; -import nova.mjs.domain.thingo.ElasticSearch.search.SearchRankingPolicy; import org.springframework.data.domain.Page; -import org.springframework.data.domain.PageImpl; import org.springframework.data.domain.Pageable; -import org.springframework.data.elasticsearch.core.SearchHit; -import org.springframework.data.elasticsearch.core.SearchHits; import org.springframework.stereotype.Service; -import java.util.List; - -/** - * 통합 검색 오케스트레이션 서비스. - * - * 역할: - * 1) Query Understanding/Rewrite 결과 생성 - * 2) Ranking policy 적용 계획 생성 - * 3) Repository 실행 및 DTO 변환 - */ @Service @RequiredArgsConstructor public class UnifiedSearchService { - private final UnifiedSearchQueryRepository unifiedSearchQueryRepository; - private final SearchIntentResolver searchIntentResolver; - private final SearchRankingPolicy searchRankingPolicy; + private final PostgresUnifiedSearchRepository postgresUnifiedSearchRepository; - /** - * 상세 검색 실행. - */ public Page search( String keyword, String type, @@ -47,121 +24,23 @@ public Page search( String normalizedType = normalizeType(type); String normalizedCategory = normalizeCategory(category); - SearchIntentContext intentContext = searchIntentResolver.resolve(keyword); - SearchQueryPlan plan = searchRankingPolicy.plan(intentContext, normalizedType, normalizedCategory, order); - - SearchHits hits = - unifiedSearchQueryRepository.search(plan, pageable); - - if (shouldFallbackToTypeKeyword(hits, normalizedType, intentContext.normalizedKeyword())) { - SearchQueryPlan fallbackPlan = withoutIntentExpansion(plan); - hits = unifiedSearchQueryRepository.search(fallbackPlan, pageable); - } - - List content = hits.getSearchHits() - .stream() - .map(this::toResponse) - .toList(); - - return new PageImpl<>(content, pageable, hits.getTotalHits()); - } - - /** - * 카테고리 상세 검색에서 결과가 비어있으면, 과도한 의도 확장을 제거한 fallback 실행 여부를 판단한다. - */ - private boolean shouldFallbackToTypeKeyword( - SearchHits hits, - String normalizedType, - String normalizedKeyword - ) { - return normalizedType != null - && normalizedKeyword != null - && !normalizedKeyword.isBlank() - && hits.getTotalHits() == 0; - } - - /** - * 의도 확장어 제약을 제거한 fallback 계획을 만든다. - */ - private SearchQueryPlan withoutIntentExpansion(SearchQueryPlan plan) { - return new SearchQueryPlan( - plan.keyword(), - plan.type(), - plan.category(), - plan.order(), - List.of(), - plan.categoryBoosts(), - plan.negativeKeywords(), - plan.negativeStrategy(), - plan.negativeDownrankBoost(), - plan.expansionTermBoost(), - plan.autocompleteBoost(), - plan.exactTitleMatchBoost(), - plan.compactTitleMatchBoost(), - plan.searchTokenMatchBoost(), - plan.typoFuzzyBoost(), - plan.categoryMatchBoost(), - plan.noticeTypeBoost(), - plan.noticeGeneralCategoryBoost(), - plan.intentRecencyWindowDays(), - plan.freshnessRules(), - plan.popularityRules() + return postgresUnifiedSearchRepository.search( + keyword, + normalizedType, + normalizedCategory, + order, + pageable ); } - /** - * ES hit -> 응답 DTO 변환. - */ - private SearchResponseDTO toResponse(SearchHit hit) { - UnifiedSearchDocument doc = hit.getContent(); - - SearchType searchType = SearchType.from(doc.getType()); - - String highlightedTitle = extractHighlight(hit, "title", doc.getTitle()); - String highlightedContent = extractHighlight(hit, "content", doc.getContent()); - - return SearchResponseDTO.builder() - .id(buildUnifiedId(searchType, doc.getOriginalId())) - .highlightedTitle(highlightedTitle) - .highlightedContent(highlightedContent) - .date(doc.getDate()) - .link(doc.getLink()) - .category(doc.getCategory()) - .type(searchType.name().toLowerCase()) - .imageUrl(doc.getImageUrl()) - .score(hit.getScore()) - .authorName(doc.getAuthorName()) - .likeCount(doc.getLikeCount()) - .commentCount(doc.getCommentCount()) - .build(); - } - - /** 통합 문서 id 규칙: TYPE:ORIGINAL_ID */ - private String buildUnifiedId(SearchType type, String originalId) { - return type.name() + ":" + originalId; - } - - /** highlight 값이 없으면 원문 fallback 반환. */ - private String extractHighlight(SearchHit hit, String field, String fallback) { - if (hit.getHighlightFields() == null) { - return fallback; - } - - List highlights = hit.getHighlightFields().get(field); - if (highlights == null || highlights.isEmpty()) { - return fallback; - } - - return highlights.get(0); - } - - /** type 파라미터를 내부 SearchType enum 값으로 정규화. */ private String normalizeType(String rawType) { + if (rawType == null || rawType.isBlank()) { + return null; + } SearchType parsed = SearchType.from(rawType); return parsed == null ? null : parsed.name(); } - /** category 파라미터를 null-safe trim 처리. */ private String normalizeCategory(String rawCategory) { if (rawCategory == null) { return null; diff --git a/src/main/java/nova/mjs/domain/thingo/realtimeKeyword/RealtimeKeywordService.java b/src/main/java/nova/mjs/domain/thingo/realtimeKeyword/RealtimeKeywordService.java index 1b0b9b47..ddc9e733 100644 --- a/src/main/java/nova/mjs/domain/thingo/realtimeKeyword/RealtimeKeywordService.java +++ b/src/main/java/nova/mjs/domain/thingo/realtimeKeyword/RealtimeKeywordService.java @@ -1,5 +1,6 @@ package nova.mjs.domain.thingo.realtimeKeyword; +import nova.mjs.domain.thingo.ElasticSearch.Repository.PostgresUnifiedSearchRepository; import org.springframework.beans.factory.annotation.Qualifier; import org.springframework.data.redis.core.RedisTemplate; import org.springframework.scheduling.annotation.Scheduled; @@ -13,9 +14,14 @@ @Service public class RealtimeKeywordService { + private final PostgresUnifiedSearchRepository postgresUnifiedSearchRepository; private final RedisTemplate redisTemplate; - public RealtimeKeywordService(@Qualifier("keywordRedisTemplate") RedisTemplate redisTemplate) { + public RealtimeKeywordService( + PostgresUnifiedSearchRepository postgresUnifiedSearchRepository, + @Qualifier("keywordRedisTemplate") RedisTemplate redisTemplate + ) { + this.postgresUnifiedSearchRepository = postgresUnifiedSearchRepository; this.redisTemplate = redisTemplate; } @@ -24,59 +30,57 @@ public RealtimeKeywordService(@Qualifier("keywordRedisTemplate") RedisTemplate getTopKeywords(int topN){ - Set keywords = redisTemplate.opsForZSet().reverseRange(ZSET_KEY, 0, topN-1); + public List getTopKeywords(int topN) { + List topKeywords = postgresUnifiedSearchRepository.topKeywords(topN); + if (!topKeywords.isEmpty()) { + return topKeywords; + } + Set keywords = redisTemplate.opsForZSet().reverseRange(ZSET_KEY, 0, topN - 1); return keywords != null ? new ArrayList<>(keywords) : Collections.emptyList(); } - //오래된 검색 기록 제거 및 점수 감소 @Scheduled(fixedRate = 60 * 60 * 1000) - public void expiredSearchRecords(){ + public void expiredSearchRecords() { long now = System.currentTimeMillis(); Set keywords = redisTemplate.opsForZSet().range(ZSET_KEY, 0, -1); - if(keywords == null || keywords.isEmpty()) return; + if (keywords == null || keywords.isEmpty()) return; - for(String keyword : keywords){ + for (String keyword : keywords) { String historyKey = LIST_KEY_PREFIX + keyword; List timestamps = redisTemplate.opsForList().range(historyKey, 0, -1); - if(timestamps == null || timestamps.isEmpty()) continue; - - boolean changed = false; + if (timestamps == null || timestamps.isEmpty()) continue; - for(String ts : new ArrayList<>(timestamps)){ + for (String ts : new ArrayList<>(timestamps)) { long time = Long.parseLong(ts); - if(now - time > ttl){ - //오래된 timestamp 제거 + if (now - time > ttl) { redisTemplate.opsForList().remove(historyKey, 1, ts); - - //점수 감소 redisTemplate.opsForZSet().incrementScore(ZSET_KEY, keyword, -1.0); - changed = true; } } Double score = redisTemplate.opsForZSet().score(ZSET_KEY, keyword); - if(score != null && score <= 0){ + if (score != null && score <= 0) { redisTemplate.opsForZSet().remove(ZSET_KEY, keyword); redisTemplate.delete(historyKey); } } } -} \ No newline at end of file +} diff --git a/src/main/resources/db/search/postgres_search_schema.sql b/src/main/resources/db/search/postgres_search_schema.sql new file mode 100644 index 00000000..c8c65456 --- /dev/null +++ b/src/main/resources/db/search/postgres_search_schema.sql @@ -0,0 +1,90 @@ +-- PostgreSQL 15+ search schema for replacing Elasticsearch unified index. +CREATE EXTENSION IF NOT EXISTS pg_trgm; +-- Optional Korean tokenizer extension (choose one by environment) +-- CREATE EXTENSION IF NOT EXISTS pg_bigm; + +CREATE TABLE IF NOT EXISTS thingo_search_document ( + id varchar(120) PRIMARY KEY, + original_id varchar(64) NOT NULL, + type varchar(40) NOT NULL, + title text, + title_normalized text, + content text, + content_normalized text, + category varchar(120), + category_normalized text, + search_tokens text, + link text, + image_url text, + created_at timestamptz, + updated_at timestamptz, + active boolean NOT NULL DEFAULT true, + popularity double precision, + like_count integer, + comment_count integer, + author_name varchar(120), + weighted_tsv tsvector +); + +CREATE INDEX IF NOT EXISTS idx_thingo_search_type_category + ON thingo_search_document(type, category); + +CREATE INDEX IF NOT EXISTS idx_thingo_search_created_at + ON thingo_search_document(created_at DESC); + +CREATE INDEX IF NOT EXISTS idx_thingo_search_weighted_tsv + ON thingo_search_document USING gin(weighted_tsv); + +CREATE INDEX IF NOT EXISTS idx_thingo_search_title_prefix + ON thingo_search_document (lower(title) text_pattern_ops); + +CREATE INDEX IF NOT EXISTS idx_thingo_search_title_trgm + ON thingo_search_document USING gin (title gin_trgm_ops); + +CREATE OR REPLACE FUNCTION update_thingo_search_tsvector() +RETURNS trigger AS $$ +BEGIN + NEW.weighted_tsv := + setweight(to_tsvector('simple', COALESCE(NEW.title, '')), 'A') + || setweight(to_tsvector('simple', COALESCE(NEW.search_tokens, '')), 'A') + || setweight(to_tsvector('simple', COALESCE(NEW.category, '')), 'B') + || setweight(to_tsvector('simple', COALESCE(NEW.content, '')), 'D'); + + NEW.updated_at := NOW(); + RETURN NEW; +END +$$ LANGUAGE plpgsql; + +DROP TRIGGER IF EXISTS trg_thingo_search_tsvector ON thingo_search_document; + +CREATE TRIGGER trg_thingo_search_tsvector +BEFORE INSERT OR UPDATE OF title, search_tokens, category, content +ON thingo_search_document +FOR EACH ROW +EXECUTE FUNCTION update_thingo_search_tsvector(); + +CREATE TABLE IF NOT EXISTS search_query_log ( + id bigserial PRIMARY KEY, + keyword varchar(120) NOT NULL, + user_id bigint, + search_type varchar(40), + category varchar(120), + searched_at timestamptz NOT NULL DEFAULT NOW() +); + +CREATE INDEX IF NOT EXISTS idx_search_query_log_keyword_time + ON search_query_log(keyword, searched_at DESC); + +CREATE INDEX IF NOT EXISTS idx_search_query_log_user_time + ON search_query_log(user_id, searched_at DESC); + +CREATE MATERIALIZED VIEW IF NOT EXISTS mv_realtime_keywords AS +SELECT keyword, + COUNT(*) AS total_count, + MAX(searched_at) AS last_searched_at +FROM search_query_log +WHERE searched_at >= NOW() - INTERVAL '3 days' +GROUP BY keyword; + +CREATE INDEX IF NOT EXISTS idx_mv_realtime_keywords_rank + ON mv_realtime_keywords(total_count DESC, last_searched_at DESC);