public inbox for [email protected]help / color / mirror / Atom feed
Re: Fix performance degradation of contended LWLock on NUMA 62+ messages / 7 participants [nested] [flat]
* Re: Fix performance degradation of contended LWLock on NUMA @ 2017-07-18 17:20 Sokolov Yura <[email protected]> 0 siblings, 2 replies; 62+ messages in thread From: Sokolov Yura @ 2017-07-18 17:20 UTC (permalink / raw) To: pgsql-hackers; +Cc: [email protected]; Andres Freund <[email protected]> On 2017-06-05 16:22, Sokolov Yura wrote: > Good day, everyone. > > This patch improves performance of contended LWLock. > Patch makes lock acquiring in single CAS loop: > 1. LWLock->state is read, and ability for lock acquiring is detected. > If there is possibility to take a lock, CAS tried. > If CAS were successful, lock is aquired (same to original version). > 2. but if lock is currently held by other backend, we check ability for > taking WaitList lock. If wait list lock is not help by anyone, CAS > perfomed for taking WaitList lock and set LW_FLAG_HAS_WAITERS at > once. > If CAS were successful, then LWLock were still held at the moment > wait > list lock were held - this proves correctness of new algorithm. And > Proc is queued to wait list then. > 3. Otherwise spin_delay is performed, and loop returns to step 1. > I'm sending rebased version with couple of one-line tweaks. (less skip_wait_list on shared lock, and don't update spin-stat on aquiring) With regards, -- Sokolov Yura aka funny_falcon Postgres Professional: https://postgrespro.ru The Russian Postgres Company -- Sent via pgsql-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers Attachments: [text/x-diff] lwlock_v2.patch (33.3K, ../../[email protected]/2-lwlock_v2.patch) download | inline diff: From dc8d9489074973ba589adf9b0239e1467cbba44b Mon Sep 17 00:00:00 2001 From: Sokolov Yura <[email protected]> Date: Mon, 29 May 2017 09:25:41 +0000 Subject: [PATCH 1/6] More correct use of spinlock inside LWLockWaitListLock. SpinDelayStatus should be function global to count iterations correctly, and produce more correct delays. Also if spin didn't spin, do not count it in spins_per_delay correction. It wasn't necessary before cause delayStatus were used only in contented cases. --- src/backend/storage/lmgr/lwlock.c | 36 ++++++++++++++++-------------------- src/backend/storage/lmgr/s_lock.c | 3 +++ 2 files changed, 19 insertions(+), 20 deletions(-) diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c index 82a1cf5150..86966b804e 100644 --- a/src/backend/storage/lmgr/lwlock.c +++ b/src/backend/storage/lmgr/lwlock.c @@ -323,6 +323,15 @@ get_lwlock_stats_entry(LWLock *lock) } return lwstats; } + +static void +add_lwlock_stats_spin_stat(LWLock *lock, SpinDelayStatus *delayStatus) +{ + lwlock_stats *lwstats; + + lwstats = get_lwlock_stats_entry(lock); + lwstats->spin_delay_count += delayStatus->delays; +} #endif /* LWLOCK_STATS */ @@ -800,13 +809,9 @@ static void LWLockWaitListLock(LWLock *lock) { uint32 old_state; -#ifdef LWLOCK_STATS - lwlock_stats *lwstats; - uint32 delays = 0; - - lwstats = get_lwlock_stats_entry(lock); -#endif + SpinDelayStatus delayStatus; + init_local_spin_delay(&delayStatus); while (true) { /* always try once to acquire lock directly */ @@ -815,20 +820,10 @@ LWLockWaitListLock(LWLock *lock) break; /* got lock */ /* and then spin without atomic operations until lock is released */ + while (old_state & LW_FLAG_LOCKED) { - SpinDelayStatus delayStatus; - - init_local_spin_delay(&delayStatus); - - while (old_state & LW_FLAG_LOCKED) - { - perform_spin_delay(&delayStatus); - old_state = pg_atomic_read_u32(&lock->state); - } -#ifdef LWLOCK_STATS - delays += delayStatus.delays; -#endif - finish_spin_delay(&delayStatus); + perform_spin_delay(&delayStatus); + old_state = pg_atomic_read_u32(&lock->state); } /* @@ -836,9 +831,10 @@ LWLockWaitListLock(LWLock *lock) * we're attempting to get it again. */ } + finish_spin_delay(&delayStatus); #ifdef LWLOCK_STATS - lwstats->spin_delay_count += delays; + add_lwlock_stats_spin_stat(lock, &delayStatus); #endif } diff --git a/src/backend/storage/lmgr/s_lock.c b/src/backend/storage/lmgr/s_lock.c index 40d8156331..f3e0fbc602 100644 --- a/src/backend/storage/lmgr/s_lock.c +++ b/src/backend/storage/lmgr/s_lock.c @@ -177,6 +177,9 @@ finish_spin_delay(SpinDelayStatus *status) if (status->cur_delay == 0) { /* we never had to delay */ + if (status->spins == 0) + /* but we didn't spin either, so ignore */ + return; if (spins_per_delay < MAX_SPINS_PER_DELAY) spins_per_delay = Min(spins_per_delay + 100, MAX_SPINS_PER_DELAY); } -- 2.11.0 From 067282cce98c77b9f1731cf7d302c37fcc70d59e Mon Sep 17 00:00:00 2001 From: Sokolov Yura <[email protected]> Date: Tue, 30 May 2017 14:05:26 +0300 Subject: [PATCH 2/6] Perform atomic LWLock acquirement or putting into wait list. Before this change, lwlock were acquired in several steps: - first try acquire lockfree (one CAS-spin-loop) - if it were not successful, queue self into wait list (two CAS-spin-loops (in fact, 3 loop, cause of atomic_fetch_and_or)) - try again to acquire lock (another one CAS loop) - if second attempt were successful, deque our self from wait queue (two CAS spin loops). With this change, lock acquired, or wait list lock acquired using single CAS loop: - if it is likely to acquire desired lock mode, we do CAS for that - otherwise we are trying for CAS to lock wait list and set necessary flag (LG_FLAG_HAS_WAITERS) at once. (LWLockWaitListUnlock still performs second CAS loop for releasing wait list lock). Correctness provided by the fact we are trying to lock wait list only when some one holds conflicting lock. If conflicting lock were released, CAS for wait list will not succeed, and we will retry to acquire lock. --- src/backend/storage/lmgr/lwlock.c | 289 ++++++++++++++++++++++---------------- 1 file changed, 171 insertions(+), 118 deletions(-) diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c index 86966b804e..c73f7430c9 100644 --- a/src/backend/storage/lmgr/lwlock.c +++ b/src/backend/storage/lmgr/lwlock.c @@ -727,20 +727,33 @@ GetLWLockIdentifier(uint32 classId, uint16 eventId) /* * Internal function that tries to atomically acquire the lwlock in the passed - * in mode. + * in mode. If it could not grab the lock, it doesn't puts proc into wait + * queue. * - * This function will not block waiting for a lock to become free - that's the - * callers job. + * It is used only in LWLockConditionalAcquire. * - * Returns true if the lock isn't free and we need to wait. + * Returns true if the lock isn't free. */ static bool -LWLockAttemptLock(LWLock *lock, LWLockMode mode) +LWLockAttemptLockOnce(LWLock *lock, LWLockMode mode) { uint32 old_state; + uint32 mask, + add; AssertArg(mode == LW_EXCLUSIVE || mode == LW_SHARED); + if (mode == LW_EXCLUSIVE) + { + mask = LW_LOCK_MASK; + add = LW_VAL_EXCLUSIVE; + } + else + { + mask = LW_VAL_EXCLUSIVE; + add = LW_VAL_SHARED; + } + /* * Read once outside the loop, later iterations will get the newer value * via compare & exchange. @@ -755,46 +768,126 @@ LWLockAttemptLock(LWLock *lock, LWLockMode mode) desired_state = old_state; - if (mode == LW_EXCLUSIVE) + lock_free = (old_state & mask) == 0; + if (lock_free) { - lock_free = (old_state & LW_LOCK_MASK) == 0; - if (lock_free) - desired_state += LW_VAL_EXCLUSIVE; + desired_state += add; + if (pg_atomic_compare_exchange_u32(&lock->state, + &old_state, desired_state)) + { + /* Great! Got the lock. */ +#ifdef LOCK_DEBUG + if (mode == LW_EXCLUSIVE) + lock->owner = MyProc; +#endif + return false; + } } else { - lock_free = (old_state & LW_VAL_EXCLUSIVE) == 0; - if (lock_free) - desired_state += LW_VAL_SHARED; + /* + * This barrier is never needed for correctness, and it is no-op + * on x86. But probably first iteration of cas loop in + * ProcArrayGroupClearXid will succeed oftener with it. + */ + pg_read_barrier(); + return true; } + } + pg_unreachable(); +} - /* - * Attempt to swap in the state we are expecting. If we didn't see - * lock to be free, that's just the old value. If we saw it as free, - * we'll attempt to mark it acquired. The reason that we always swap - * in the value is that this doubles as a memory barrier. We could try - * to be smarter and only swap in values if we saw the lock as free, - * but benchmark haven't shown it as beneficial so far. - * - * Retry if the value changed since we last looked at it. - */ - if (pg_atomic_compare_exchange_u32(&lock->state, - &old_state, desired_state)) +static void LWLockQueueSelfLocked(LWLock *lock, LWLockMode mode); + +/* + * Internal function that tries to atomically acquire the lwlock in the passed + * in mode or put it self into waiting queue with waitmode. + * + * This function will not block waiting for a lock to become free - that's the + * callers job. + * + * Returns true if the lock isn't free and we are in a wait queue. + */ +static inline bool +LWLockAttemptLockOrQueueSelf(LWLock *lock, LWLockMode mode, LWLockMode waitmode) +{ + uint32 old_state; + SpinDelayStatus delayStatus; + bool lock_free; + uint32 mask, + add; + + AssertArg(mode == LW_EXCLUSIVE || mode == LW_SHARED); + + if (mode == LW_EXCLUSIVE) + { + mask = LW_LOCK_MASK; + add = LW_VAL_EXCLUSIVE; + } + else + { + mask = LW_VAL_EXCLUSIVE; + add = LW_VAL_SHARED; + } + + init_local_spin_delay(&delayStatus); + + /* + * Read once outside the loop. Later iterations will get the newer value + * either via compare & exchange or with read after perform_spin_delay. + */ + old_state = pg_atomic_read_u32(&lock->state); + /* loop until we've determined whether we could acquire the lock or not */ + while (true) + { + uint32 desired_state; + + desired_state = old_state; + + lock_free = (old_state & mask) == 0; + if (lock_free) { - if (lock_free) + desired_state += add; + if (pg_atomic_compare_exchange_u32(&lock->state, + &old_state, desired_state)) { /* Great! Got the lock. */ #ifdef LOCK_DEBUG if (mode == LW_EXCLUSIVE) lock->owner = MyProc; #endif - return false; + break; + } + } + else if ((old_state & LW_FLAG_LOCKED) == 0) + { + desired_state |= LW_FLAG_LOCKED | LW_FLAG_HAS_WAITERS; + if (pg_atomic_compare_exchange_u32(&lock->state, + &old_state, desired_state)) + { + LWLockQueueSelfLocked(lock, waitmode); + break; } - else - return true; /* somebody else has the lock */ + } + else + { + perform_spin_delay(&delayStatus); + old_state = pg_atomic_read_u32(&lock->state); } } - pg_unreachable(); +#ifdef LWLOCK_STATS + add_lwlock_stats_spin_stat(lock, &delayStatus); +#endif + + /* + * We intentionally do not call finish_spin_delay here, cause loop above + * usually finished in queueing into wait list on contention, and doesn't + * reach spins_per_delay (and so, doesn't sleep inside of + * perform_spin_delay). Also, different LWLocks has very different + * contention pattern, and it is wrong to update spin-lock statistic based + * on LWLock contention. + */ + return !lock_free; } /* @@ -960,7 +1053,7 @@ LWLockWakeup(LWLock *lock) } /* - * Add ourselves to the end of the queue. + * Add ourselves to the end of the queue, acquiring waitlist lock. * * NB: Mode can be LW_WAIT_UNTIL_FREE here! */ @@ -983,6 +1076,19 @@ LWLockQueueSelf(LWLock *lock, LWLockMode mode) /* setting the flag is protected by the spinlock */ pg_atomic_fetch_or_u32(&lock->state, LW_FLAG_HAS_WAITERS); + LWLockQueueSelfLocked(lock, mode); +} + +/* + * Add ourselves to the end of the queue. + * Lock should be held at this moment, and all neccessary flags are already + * set. It will release wait list lock. + * + * NB: Mode can be LW_WAIT_UNTIL_FREE here! + */ +static void +LWLockQueueSelfLocked(LWLock *lock, LWLockMode mode) +{ MyProc->lwWaiting = true; MyProc->lwWaitMode = mode; @@ -1165,11 +1271,7 @@ LWLockAcquire(LWLock *lock, LWLockMode mode) { bool mustwait; - /* - * Try to grab the lock the first time, we're not in the waitqueue - * yet/anymore. - */ - mustwait = LWLockAttemptLock(lock, mode); + mustwait = LWLockAttemptLockOrQueueSelf(lock, mode, mode); if (!mustwait) { @@ -1178,42 +1280,17 @@ LWLockAcquire(LWLock *lock, LWLockMode mode) } /* - * Ok, at this point we couldn't grab the lock on the first try. We - * cannot simply queue ourselves to the end of the list and wait to be - * woken up because by now the lock could long have been released. - * Instead add us to the queue and try to grab the lock again. If we - * succeed we need to revert the queuing and be happy, otherwise we - * recheck the lock. If we still couldn't grab it, we know that the - * other locker will see our queue entries when releasing since they - * existed before we checked for the lock. - */ - - /* add to the queue */ - LWLockQueueSelf(lock, mode); - - /* we're now guaranteed to be woken up if necessary */ - mustwait = LWLockAttemptLock(lock, mode); - - /* ok, grabbed the lock the second time round, need to undo queueing */ - if (!mustwait) - { - LOG_LWDEBUG("LWLockAcquire", lock, "acquired, undoing queue"); - - LWLockDequeueSelf(lock); - break; - } - - /* * Wait until awakened. * - * Since we share the process wait semaphore with the regular lock - * manager and ProcWaitForSignal, and we may need to acquire an LWLock - * while one of those is pending, it is possible that we get awakened - * for a reason other than being signaled by LWLockRelease. If so, - * loop back and wait again. Once we've gotten the LWLock, - * re-increment the sema by the number of additional signals received, - * so that the lock manager or signal manager will see the received - * signal when it next waits. + * At this point we are already in a wait queue. Since we share the + * process wait semaphore with the regular lock manager and + * ProcWaitForSignal, and we may need to acquire an LWLock while one + * of those is pending, it is possible that we get awakened for a + * reason other than being signaled by LWLockRelease. If so, loop back + * and wait again. Once we've gotten the LWLock, re-increment the + * sema by the number of additional signals received, so that the lock + * manager or signal manager will see the received signal when it next + * waits. */ LOG_LWDEBUG("LWLockAcquire", lock, "waiting"); @@ -1296,7 +1373,7 @@ LWLockConditionalAcquire(LWLock *lock, LWLockMode mode) HOLD_INTERRUPTS(); /* Check for the lock */ - mustwait = LWLockAttemptLock(lock, mode); + mustwait = LWLockAttemptLockOnce(lock, mode); if (mustwait) { @@ -1357,67 +1434,43 @@ LWLockAcquireOrWait(LWLock *lock, LWLockMode mode) */ HOLD_INTERRUPTS(); - /* - * NB: We're using nearly the same twice-in-a-row lock acquisition - * protocol as LWLockAcquire(). Check its comments for details. - */ - mustwait = LWLockAttemptLock(lock, mode); + mustwait = LWLockAttemptLockOrQueueSelf(lock, mode, LW_WAIT_UNTIL_FREE); if (mustwait) { - LWLockQueueSelf(lock, LW_WAIT_UNTIL_FREE); - - mustwait = LWLockAttemptLock(lock, mode); - - if (mustwait) - { - /* - * Wait until awakened. Like in LWLockAcquire, be prepared for - * bogus wakeups, because we share the semaphore with - * ProcWaitForSignal. - */ - LOG_LWDEBUG("LWLockAcquireOrWait", lock, "waiting"); + /* + * Wait until awakened. Like in LWLockAcquire, be prepared for bogus + * wakeups, because we share the semaphore with ProcWaitForSignal. + */ + LOG_LWDEBUG("LWLockAcquireOrWait", lock, "waiting"); #ifdef LWLOCK_STATS - lwstats->block_count++; + lwstats->block_count++; #endif - LWLockReportWaitStart(lock); - TRACE_POSTGRESQL_LWLOCK_WAIT_START(T_NAME(lock), mode); + LWLockReportWaitStart(lock); + TRACE_POSTGRESQL_LWLOCK_WAIT_START(T_NAME(lock), mode); - for (;;) - { - PGSemaphoreLock(proc->sem); - if (!proc->lwWaiting) - break; - extraWaits++; - } + for (;;) + { + PGSemaphoreLock(proc->sem); + if (!proc->lwWaiting) + break; + extraWaits++; + } #ifdef LOCK_DEBUG - { - /* not waiting anymore */ - uint32 nwaiters PG_USED_FOR_ASSERTS_ONLY = pg_atomic_fetch_sub_u32(&lock->nwaiters, 1); - - Assert(nwaiters < MAX_BACKENDS); - } -#endif - TRACE_POSTGRESQL_LWLOCK_WAIT_DONE(T_NAME(lock), mode); - LWLockReportWaitEnd(); - - LOG_LWDEBUG("LWLockAcquireOrWait", lock, "awakened"); - } - else { - LOG_LWDEBUG("LWLockAcquireOrWait", lock, "acquired, undoing queue"); + /* not waiting anymore */ + uint32 nwaiters PG_USED_FOR_ASSERTS_ONLY = pg_atomic_fetch_sub_u32(&lock->nwaiters, 1); - /* - * Got lock in the second attempt, undo queueing. We need to treat - * this as having successfully acquired the lock, otherwise we'd - * not necessarily wake up people we've prevented from acquiring - * the lock. - */ - LWLockDequeueSelf(lock); + Assert(nwaiters < MAX_BACKENDS); } +#endif + TRACE_POSTGRESQL_LWLOCK_WAIT_DONE(T_NAME(lock), mode); + LWLockReportWaitEnd(); + + LOG_LWDEBUG("LWLockAcquireOrWait", lock, "awakened"); } /* -- 2.11.0 From 4bf5f8a3739cc32b5a996188e8766e7a472f2647 Mon Sep 17 00:00:00 2001 From: Sokolov Yura <[email protected]> Date: Tue, 30 May 2017 17:21:48 +0300 Subject: [PATCH 3/6] use custom loop for LWLockWaitForVar Previously LWLockWaitForVar used following procedure: - check lock state and var value LWLockConflictsWithVar - LWLockQueueSelf - recheck lock state and var with LWLockConflictsWithVar - LWLockDequeueSelf if lock were released by holder or variable changed. With this change, LWLockConflictsWithVar performs single atomic: - check lock state -- if lock is not held, return "no need to wait" -- otherwise try to lock wait list - with lock on wait list check value of variable (return if it was changed) - set flags on a state and queue self into wait list -- if CAS for flags failed, recheck if lock still held. If it is not held, return. With this change there is no need it LWLockQueueSelf and LWLockDequequeSelf any more, so this functions are removed. --- src/backend/storage/lmgr/lwlock.c | 251 +++++++++++--------------------------- 1 file changed, 70 insertions(+), 181 deletions(-) diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c index c73f7430c9..487824ea9f 100644 --- a/src/backend/storage/lmgr/lwlock.c +++ b/src/backend/storage/lmgr/lwlock.c @@ -176,7 +176,6 @@ typedef struct lwlock_stats int sh_acquire_count; int ex_acquire_count; int block_count; - int dequeue_self_count; int spin_delay_count; } lwlock_stats; @@ -284,11 +283,11 @@ print_lwlock_stats(int code, Datum arg) while ((lwstats = (lwlock_stats *) hash_seq_search(&scan)) != NULL) { fprintf(stderr, - "PID %d lwlock %s %p: shacq %u exacq %u blk %u spindelay %u dequeue self %u\n", + "PID %d lwlock %s %p: shacq %u exacq %u blk %u spindelay %u\n", MyProcPid, LWLockTrancheArray[lwstats->key.tranche], lwstats->key.instance, lwstats->sh_acquire_count, lwstats->ex_acquire_count, lwstats->block_count, - lwstats->spin_delay_count, lwstats->dequeue_self_count); + lwstats->spin_delay_count); } LWLockRelease(&MainLWLockArray[0].lock); @@ -318,7 +317,6 @@ get_lwlock_stats_entry(LWLock *lock) lwstats->sh_acquire_count = 0; lwstats->ex_acquire_count = 0; lwstats->block_count = 0; - lwstats->dequeue_self_count = 0; lwstats->spin_delay_count = 0; } return lwstats; @@ -1053,33 +1051,6 @@ LWLockWakeup(LWLock *lock) } /* - * Add ourselves to the end of the queue, acquiring waitlist lock. - * - * NB: Mode can be LW_WAIT_UNTIL_FREE here! - */ -static void -LWLockQueueSelf(LWLock *lock, LWLockMode mode) -{ - /* - * If we don't have a PGPROC structure, there's no way to wait. This - * should never occur, since MyProc should only be null during shared - * memory initialization. - */ - if (MyProc == NULL) - elog(PANIC, "cannot wait without a PGPROC structure"); - - if (MyProc->lwWaiting) - elog(PANIC, "queueing for lock while waiting on another one"); - - LWLockWaitListLock(lock); - - /* setting the flag is protected by the spinlock */ - pg_atomic_fetch_or_u32(&lock->state, LW_FLAG_HAS_WAITERS); - - LWLockQueueSelfLocked(lock, mode); -} - -/* * Add ourselves to the end of the queue. * Lock should be held at this moment, and all neccessary flags are already * set. It will release wait list lock. @@ -1108,100 +1079,6 @@ LWLockQueueSelfLocked(LWLock *lock, LWLockMode mode) } /* - * Remove ourselves from the waitlist. - * - * This is used if we queued ourselves because we thought we needed to sleep - * but, after further checking, we discovered that we don't actually need to - * do so. - */ -static void -LWLockDequeueSelf(LWLock *lock) -{ - bool found = false; - proclist_mutable_iter iter; - -#ifdef LWLOCK_STATS - lwlock_stats *lwstats; - - lwstats = get_lwlock_stats_entry(lock); - - lwstats->dequeue_self_count++; -#endif - - LWLockWaitListLock(lock); - - /* - * Can't just remove ourselves from the list, but we need to iterate over - * all entries as somebody else could have dequeued us. - */ - proclist_foreach_modify(iter, &lock->waiters, lwWaitLink) - { - if (iter.cur == MyProc->pgprocno) - { - found = true; - proclist_delete(&lock->waiters, iter.cur, lwWaitLink); - break; - } - } - - if (proclist_is_empty(&lock->waiters) && - (pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS) != 0) - { - pg_atomic_fetch_and_u32(&lock->state, ~LW_FLAG_HAS_WAITERS); - } - - /* XXX: combine with fetch_and above? */ - LWLockWaitListUnlock(lock); - - /* clear waiting state again, nice for debugging */ - if (found) - MyProc->lwWaiting = false; - else - { - int extraWaits = 0; - - /* - * Somebody else dequeued us and has or will wake us up. Deal with the - * superfluous absorption of a wakeup. - */ - - /* - * Reset releaseOk if somebody woke us before we removed ourselves - - * they'll have set it to false. - */ - pg_atomic_fetch_or_u32(&lock->state, LW_FLAG_RELEASE_OK); - - /* - * Now wait for the scheduled wakeup, otherwise our ->lwWaiting would - * get reset at some inconvenient point later. Most of the time this - * will immediately return. - */ - for (;;) - { - PGSemaphoreLock(MyProc->sem); - if (!MyProc->lwWaiting) - break; - extraWaits++; - } - - /* - * Fix the process wait semaphore's count for any absorbed wakeups. - */ - while (extraWaits-- > 0) - PGSemaphoreUnlock(MyProc->sem); - } - -#ifdef LOCK_DEBUG - { - /* not waiting anymore */ - uint32 nwaiters PG_USED_FOR_ASSERTS_ONLY = pg_atomic_fetch_sub_u32(&lock->nwaiters, 1); - - Assert(nwaiters < MAX_BACKENDS); - } -#endif -} - -/* * LWLockAcquire - acquire a lightweight lock in the specified mode * * If the lock is not available, sleep until it is. Returns true if the lock @@ -1514,43 +1391,87 @@ LWLockConflictsWithVar(LWLock *lock, { bool mustwait; uint64 value; + uint32 state; + SpinDelayStatus delayStatus; - /* - * Test first to see if it the slot is free right now. - * - * XXX: the caller uses a spinlock before this, so we don't need a memory - * barrier here as far as the current usage is concerned. But that might - * not be safe in general. - */ - mustwait = (pg_atomic_read_u32(&lock->state) & LW_VAL_EXCLUSIVE) != 0; - - if (!mustwait) - { - *result = true; - return false; - } - - *result = false; + init_local_spin_delay(&delayStatus); /* + * We are trying to detect exclusive lock on state, or value change. If + * neither happens, we put ourself into wait queue. + * * Read value using the lwlock's wait list lock, as we can't generally * rely on atomic 64 bit reads/stores. TODO: On platforms with a way to * do atomic 64 bit reads/writes the spinlock should be optimized away. */ - LWLockWaitListLock(lock); - value = *valptr; - LWLockWaitListUnlock(lock); + while (true) + { + state = pg_atomic_read_u32(&lock->state); + mustwait = (state & LW_VAL_EXCLUSIVE) != 0; - if (value != oldval) + if (!mustwait) + { + *result = true; + goto exit; + } + + if ((state & LW_FLAG_LOCKED) == 0) + { + if (pg_atomic_compare_exchange_u32(&lock->state, &state, + state | LW_FLAG_LOCKED)) + break; + } + perform_spin_delay(&delayStatus); + } + + /* Now we've locked wait queue, and someone holds EXCLUSIVE lock. */ + value = *valptr; + mustwait = value == oldval; + if (!mustwait) { - mustwait = false; + /* value changed, no need to block */ + LWLockWaitListUnlock(lock); *newval = value; + goto exit; } - else + + /* + * If value didn't change, we have to set LW_FLAG_HAS_WAITERS and + * LW_FLAG_RELEASE_OK flags and put ourself into wait queue. We could find + * that the lock is not held anymore, in this case we should return + * without blocking. + * + * Note: value could not change again cause we are holding WaitList lock. + */ + while (true) { - mustwait = true; - } + uint32 desired_state; + desired_state = state | LW_FLAG_HAS_WAITERS | LW_FLAG_RELEASE_OK; + if (pg_atomic_compare_exchange_u32(&lock->state, &state, + desired_state)) + { + /* + * we satisfy invariant: flags should be set while someone holds + * the lock + */ + LWLockQueueSelfLocked(lock, LW_WAIT_UNTIL_FREE); + break; + } + if ((state & LW_VAL_EXCLUSIVE) == 0) + { + /* lock was released, no need to block anymore */ + LWLockWaitListUnlock(lock); + *result = true; + mustwait = false; + break; + } + } +exit: +#ifdef LWLOCK_STATS + add_lwlock_stats_spin_stat(lock, &delayStatus); +#endif + finish_spin_delay(&delayStatus); return mustwait; } @@ -1602,38 +1523,6 @@ LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval) break; /* the lock was free or value didn't match */ /* - * Add myself to wait queue. Note that this is racy, somebody else - * could wakeup before we're finished queuing. NB: We're using nearly - * the same twice-in-a-row lock acquisition protocol as - * LWLockAcquire(). Check its comments for details. The only - * difference is that we also have to check the variable's values when - * checking the state of the lock. - */ - LWLockQueueSelf(lock, LW_WAIT_UNTIL_FREE); - - /* - * Set RELEASE_OK flag, to make sure we get woken up as soon as the - * lock is released. - */ - pg_atomic_fetch_or_u32(&lock->state, LW_FLAG_RELEASE_OK); - - /* - * We're now guaranteed to be woken up if necessary. Recheck the lock - * and variables state. - */ - mustwait = LWLockConflictsWithVar(lock, valptr, oldval, newval, - &result); - - /* Ok, no conflict after we queued ourselves. Undo queueing. */ - if (!mustwait) - { - LOG_LWDEBUG("LWLockWaitForVar", lock, "free, undoing queue"); - - LWLockDequeueSelf(lock); - break; - } - - /* * Wait until awakened. * * Since we share the process wait semaphore with the regular lock -- 2.11.0 From c6ea4c993b2e6951c724abc7a9d3094d942adbcb Mon Sep 17 00:00:00 2001 From: Sokolov Yura <[email protected]> Date: Wed, 31 May 2017 13:56:30 +0000 Subject: [PATCH 4/6] Make acquiring LWLock to look more like spinlock. If delayStatus.spins <= spin_per_delay/8, then do not try to add itself into wait list, but instead do perform_spin_delay and retry lock acquiring. Since delayStatus.spins is reset after each pg_usleep, it spins after sleep again. It greatly improves performance on "contended but not much" cases, and overall performance also. --- src/backend/storage/lmgr/lwlock.c | 10 ++++++++-- src/backend/storage/lmgr/s_lock.c | 2 +- src/include/storage/s_lock.h | 2 ++ 3 files changed, 11 insertions(+), 3 deletions(-) diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c index 487824ea9f..50e0d53ec6 100644 --- a/src/backend/storage/lmgr/lwlock.c +++ b/src/backend/storage/lmgr/lwlock.c @@ -814,6 +814,7 @@ LWLockAttemptLockOrQueueSelf(LWLock *lock, LWLockMode mode, LWLockMode waitmode) bool lock_free; uint32 mask, add; + int skip_wait_list; AssertArg(mode == LW_EXCLUSIVE || mode == LW_SHARED); @@ -821,11 +822,13 @@ LWLockAttemptLockOrQueueSelf(LWLock *lock, LWLockMode mode, LWLockMode waitmode) { mask = LW_LOCK_MASK; add = LW_VAL_EXCLUSIVE; + skip_wait_list = spins_per_delay / 8; } else { mask = LW_VAL_EXCLUSIVE; add = LW_VAL_SHARED; + skip_wait_list = spins_per_delay / 4; } init_local_spin_delay(&delayStatus); @@ -857,7 +860,7 @@ LWLockAttemptLockOrQueueSelf(LWLock *lock, LWLockMode mode, LWLockMode waitmode) break; } } - else if ((old_state & LW_FLAG_LOCKED) == 0) + else if (delayStatus.spins > skip_wait_list && (old_state & LW_FLAG_LOCKED) == 0) { desired_state |= LW_FLAG_LOCKED | LW_FLAG_HAS_WAITERS; if (pg_atomic_compare_exchange_u32(&lock->state, @@ -1393,9 +1396,12 @@ LWLockConflictsWithVar(LWLock *lock, uint64 value; uint32 state; SpinDelayStatus delayStatus; + int skip_wait_list; init_local_spin_delay(&delayStatus); + skip_wait_list = spins_per_delay / 4; + /* * We are trying to detect exclusive lock on state, or value change. If * neither happens, we put ourself into wait queue. @@ -1415,7 +1421,7 @@ LWLockConflictsWithVar(LWLock *lock, goto exit; } - if ((state & LW_FLAG_LOCKED) == 0) + if (delayStatus.spins > skip_wait_list && (state & LW_FLAG_LOCKED) == 0) { if (pg_atomic_compare_exchange_u32(&lock->state, &state, state | LW_FLAG_LOCKED)) diff --git a/src/backend/storage/lmgr/s_lock.c b/src/backend/storage/lmgr/s_lock.c index f3e0fbc602..ce1f08656c 100644 --- a/src/backend/storage/lmgr/s_lock.c +++ b/src/backend/storage/lmgr/s_lock.c @@ -63,7 +63,7 @@ slock_t dummy_spinlock; -static int spins_per_delay = DEFAULT_SPINS_PER_DELAY; +int spins_per_delay = DEFAULT_SPINS_PER_DELAY; /* diff --git a/src/include/storage/s_lock.h b/src/include/storage/s_lock.h index bbf505e246..1dd4c7880d 100644 --- a/src/include/storage/s_lock.h +++ b/src/include/storage/s_lock.h @@ -964,6 +964,8 @@ extern int s_lock(volatile slock_t *lock, const char *file, int line, const char /* Support for dynamic adjustment of spins_per_delay */ #define DEFAULT_SPINS_PER_DELAY 100 +/* Read in lwlock.c to adjust behavior */ +extern int spins_per_delay; extern void set_spins_per_delay(int shared_spins_per_delay); extern int update_spins_per_delay(int shared_spins_per_delay); -- 2.11.0 From 877001bc013e8426b3cfdc9c28434f8c9a99b1da Mon Sep 17 00:00:00 2001 From: Sokolov Yura <[email protected]> Date: Tue, 30 May 2017 18:54:25 +0300 Subject: [PATCH 5/6] Fix implementation description in a lwlock.c . --- src/backend/storage/lmgr/lwlock.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c index 50e0d53ec6..d38d031d90 100644 --- a/src/backend/storage/lmgr/lwlock.c +++ b/src/backend/storage/lmgr/lwlock.c @@ -62,16 +62,15 @@ * notice that we have to wait. Unfortunately by the time we have finished * queuing, the former locker very well might have already finished it's * work. That's problematic because we're now stuck waiting inside the OS. - - * To mitigate those races we use a two phased attempt at locking: - * Phase 1: Try to do it atomically, if we succeed, nice - * Phase 2: Add ourselves to the waitqueue of the lock - * Phase 3: Try to grab the lock again, if we succeed, remove ourselves from - * the queue - * Phase 4: Sleep till wake-up, goto Phase 1 * - * This protects us against the problem from above as nobody can release too - * quick, before we're queued, since after Phase 2 we're already queued. + * This race is avoiding by taking lock for wait list using CAS with a old + * state value, so it could succeed only if lock is still held. Necessary flag + * is set with same CAS. + * + * Inside LWLockConflictsWithVar wait list lock is reused to protect variable + * value. So first it is acquired to check variable value, then flags are + * set with second CAS before queueing to wait list to ensure lock were not + * released yet. * ------------------------------------------------------------------------- */ #include "postgres.h" -- 2.11.0 From 1dacce7f45edc4f234266665ba3ce88a28588fda Mon Sep 17 00:00:00 2001 From: Sokolov Yura <[email protected]> Date: Fri, 2 Jun 2017 11:34:23 +0000 Subject: [PATCH 6/6] Make SpinDelayStatus a bit lighter. It saves couple of instruction of fast path of spin-loop, and so makes fast path of LWLock a bit faster (and in other places where spinlock is used). Also it makes call to perform_spin_delay a bit slower, that could positively affect on spin behavior, especially if no `pause` instruction present. --- src/backend/storage/lmgr/s_lock.c | 9 +++++---- src/include/storage/s_lock.h | 16 ++++++---------- 2 files changed, 11 insertions(+), 14 deletions(-) diff --git a/src/backend/storage/lmgr/s_lock.c b/src/backend/storage/lmgr/s_lock.c index ce1f08656c..11c0720951 100644 --- a/src/backend/storage/lmgr/s_lock.c +++ b/src/backend/storage/lmgr/s_lock.c @@ -93,11 +93,11 @@ s_lock(volatile slock_t *lock, const char *file, int line, const char *func) { SpinDelayStatus delayStatus; - init_spin_delay(&delayStatus, file, line, func); + init_spin_delay(&delayStatus); while (TAS_SPIN(lock)) { - perform_spin_delay(&delayStatus); + perform_spin_delay_with_info(&delayStatus, file, line, func); } finish_spin_delay(&delayStatus); @@ -122,7 +122,8 @@ s_unlock(volatile slock_t *lock) * Wait while spinning on a contended spinlock. */ void -perform_spin_delay(SpinDelayStatus *status) +perform_spin_delay_with_info(SpinDelayStatus *status, + const char *file, int line, const char *func) { /* CPU-specific delay each time through the loop */ SPIN_DELAY(); @@ -131,7 +132,7 @@ perform_spin_delay(SpinDelayStatus *status) if (++(status->spins) >= spins_per_delay) { if (++(status->delays) > NUM_DELAYS) - s_lock_stuck(status->file, status->line, status->func); + s_lock_stuck(file, line, func); if (status->cur_delay == 0) /* first time to delay? */ status->cur_delay = MIN_DELAY_USEC; diff --git a/src/include/storage/s_lock.h b/src/include/storage/s_lock.h index 1dd4c7880d..e5c4645461 100644 --- a/src/include/storage/s_lock.h +++ b/src/include/storage/s_lock.h @@ -979,25 +979,21 @@ typedef struct int spins; int delays; int cur_delay; - const char *file; - int line; - const char *func; } SpinDelayStatus; static inline void -init_spin_delay(SpinDelayStatus *status, - const char *file, int line, const char *func) +init_spin_delay(SpinDelayStatus *status) { status->spins = 0; status->delays = 0; status->cur_delay = 0; - status->file = file; - status->line = line; - status->func = func; } -#define init_local_spin_delay(status) init_spin_delay(status, __FILE__, __LINE__, PG_FUNCNAME_MACRO) -void perform_spin_delay(SpinDelayStatus *status); +#define init_local_spin_delay(status) init_spin_delay(status) +void perform_spin_delay_with_info(SpinDelayStatus *status, + const char *file, int line, const char *func); +#define perform_spin_delay(status) \ + perform_spin_delay_with_info(status, __FILE__, __LINE__, PG_FUNCNAME_MACRO) void finish_spin_delay(SpinDelayStatus *status); #endif /* S_LOCK_H */ -- 2.11.0 ^ permalink raw reply [nested|flat] 62+ messages in thread
* Re: Fix performance degradation of contended LWLock on NUMA @ 2017-08-10 12:58 Sokolov Yura <[email protected]> parent: Sokolov Yura <[email protected]> 1 sibling, 0 replies; 62+ messages in thread From: Sokolov Yura @ 2017-08-10 12:58 UTC (permalink / raw) To: pgsql-hackers; +Cc: [email protected]; Andres Freund <[email protected]> On 2017-07-18 20:20, Sokolov Yura wrote: > On 2017-06-05 16:22, Sokolov Yura wrote: >> Good day, everyone. >> >> This patch improves performance of contended LWLock. >> Patch makes lock acquiring in single CAS loop: >> 1. LWLock->state is read, and ability for lock acquiring is detected. >> If there is possibility to take a lock, CAS tried. >> If CAS were successful, lock is aquired (same to original version). >> 2. but if lock is currently held by other backend, we check ability >> for >> taking WaitList lock. If wait list lock is not help by anyone, CAS >> perfomed for taking WaitList lock and set LW_FLAG_HAS_WAITERS at >> once. >> If CAS were successful, then LWLock were still held at the moment >> wait >> list lock were held - this proves correctness of new algorithm. And >> Proc is queued to wait list then. >> 3. Otherwise spin_delay is performed, and loop returns to step 1. >> > > I'm sending rebased version with couple of one-line tweaks. > (less skip_wait_list on shared lock, and don't update spin-stat on > aquiring) > > With regards, Here are results for zipfian distribution (50/50 r/w) in conjunction with "lazy hash table for XidInMVCCSnapshot": (https://www.postgresql.org/message-id/642da34694800dab801f04c62950ce8a%40postgrespro.ru) clients | master | hashsnap2 | hashsnap2_lwlock --------+--------+-----------+------------------ 10 | 203384 | 203813 | 204852 20 | 334344 | 334268 | 363510 40 | 228496 | 231777 | 383820 70 | 146892 | 148173 | 221326 110 | 99741 | 111580 | 157327 160 | 65257 | 81230 | 112028 230 | 38344 | 56790 | 77514 310 | 22355 | 39249 | 55907 400 | 13402 | 26899 | 39742 500 | 8382 | 17855 | 28362 650 | 5313 | 11450 | 17497 800 | 3352 | 7816 | 11030 With regards, -- Sokolov Yura aka funny_falcon Postgres Professional: https://postgrespro.ru The Russian Postgres Company -- Sent via pgsql-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers ^ permalink raw reply [nested|flat] 62+ messages in thread
* Re: Fix performance degradation of contended LWLock on NUMA @ 2017-09-08 15:33 Jesper Pedersen <[email protected]> parent: Sokolov Yura <[email protected]> 1 sibling, 1 reply; 62+ messages in thread From: Jesper Pedersen @ 2017-09-08 15:33 UTC (permalink / raw) To: Sokolov Yura <[email protected]>; +Cc: pgsql-hackers; Andres Freund <[email protected]> Hi, On 07/18/2017 01:20 PM, Sokolov Yura wrote: > I'm sending rebased version with couple of one-line tweaks. > (less skip_wait_list on shared lock, and don't update spin-stat on > aquiring) > I have been running this patch on a 2S/28C/56T/256Gb w/ 2 x RAID10 SSD setup (1 to 375 clients on logged tables). I'm seeing -M prepared: Up to 11% improvement -M prepared -S: No improvement, no regression ("noise") -M prepared -N: Up to 12% improvement for all runs the improvement shows up the closer you get to the number of CPU threads, or above. Although I'm not seeing the same improvements as you on very large client counts there are definitely improvements :) Some comments: ============== lwlock.c: --------- + * This race is avoiding by taking lock for wait list using CAS with a old + * state value, so it could succeed only if lock is still held. Necessary flag + * is set with same CAS. + * + * Inside LWLockConflictsWithVar wait list lock is reused to protect variable + * value. So first it is acquired to check variable value, then flags are + * set with second CAS before queueing to wait list to ensure lock were not + * released yet. * This race is avoided by taking a lock for the wait list using CAS with the old * state value, so it would only succeed if lock is still held. Necessary flag * is set using the same CAS. * * Inside LWLockConflictsWithVar the wait list lock is reused to protect the variable * value. First the lock is acquired to check the variable value, then flags are * set with a second CAS before queuing to the wait list in order to ensure that the lock was not * released yet. +static void +add_lwlock_stats_spin_stat(LWLock *lock, SpinDelayStatus *delayStatus) Add a method description. + /* + * This barrier is never needed for correctness, and it is no-op + * on x86. But probably first iteration of cas loop in + * ProcArrayGroupClearXid will succeed oftener with it. + */ * "more often" +static inline bool +LWLockAttemptLockOrQueueSelf(LWLock *lock, LWLockMode mode, LWLockMode waitmode) I'll leave it to the Committer to decide if this method is too big to be "inline". + /* + * We intentionally do not call finish_spin_delay here, cause loop above + * usually finished in queueing into wait list on contention, and doesn't + * reach spins_per_delay (and so, doesn't sleep inside of + * perform_spin_delay). Also, different LWLocks has very different + * contention pattern, and it is wrong to update spin-lock statistic based + * on LWLock contention. + */ /* * We intentionally do not call finish_spin_delay here, because the loop above * usually finished by queuing into the wait list on contention, and doesn't * reach spins_per_delay thereby doesn't sleep inside of * perform_spin_delay. Also, different LWLocks has very different * contention pattern, and it is wrong to update spin-lock statistic based * on LWLock contention. */ s_lock.c: --------- + if (status->spins == 0) + /* but we didn't spin either, so ignore */ + return; Use { } for the if, or move the comment out of the nesting for readability. Open questions: --------------- * spins_per_delay as extern * Calculation of skip_wait_list You could run the patch through pgindent too. Passes make check-world. Status: Waiting on Author Best regards, Jesper -- Sent via pgsql-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers ^ permalink raw reply [nested|flat] 62+ messages in thread
* Re: Fix performance degradation of contended LWLock on NUMA @ 2017-09-08 19:35 Sokolov Yura <[email protected]> parent: Jesper Pedersen <[email protected]> 0 siblings, 2 replies; 62+ messages in thread From: Sokolov Yura @ 2017-09-08 19:35 UTC (permalink / raw) To: [email protected]; +Cc: pgsql-hackers; Andres Freund <[email protected]>; [email protected] Hi, Jesper Thank you for reviewing. On 2017-09-08 18:33, Jesper Pedersen wrote: > Hi, > > On 07/18/2017 01:20 PM, Sokolov Yura wrote: >> I'm sending rebased version with couple of one-line tweaks. >> (less skip_wait_list on shared lock, and don't update spin-stat on >> aquiring) >> > > I have been running this patch on a 2S/28C/56T/256Gb w/ 2 x RAID10 SSD > setup (1 to 375 clients on logged tables). > > I'm seeing > > -M prepared: Up to 11% improvement > -M prepared -S: No improvement, no regression ("noise") > -M prepared -N: Up to 12% improvement > > for all runs the improvement shows up the closer you get to the number > of CPU threads, or above. Although I'm not seeing the same > improvements as you on very large client counts there are definitely > improvements :) It is expected: - patch "fixes NUMA": for example, it doesn't give improvement on 1 socket at all (I've tested it using numactl to bind to 1 socket) - and certainly it gives less improvement on 2 sockets than on 4 sockets (and 28 cores vs 72 cores also gives difference), - one of hot points were CLogControlLock, and it were fixed with "Group mode for CLOG updates" [1] > > Some comments: > ============== > > lwlock.c: > --------- > > ... > > +static void > +add_lwlock_stats_spin_stat(LWLock *lock, SpinDelayStatus *delayStatus) > > Add a method description. Neighbor functions above also has no description, that is why I didn't add. But ok, I've added now. > > +static inline bool > +LWLockAttemptLockOrQueueSelf(LWLock *lock, LWLockMode mode, > LWLockMode waitmode) > > I'll leave it to the Committer to decide if this method is too big to > be "inline". GCC 4.9 doesn't want to inline it without directive, and function call is then remarkable in profile. Attach contains version with all suggestions applied except remove of "inline". > Open questions: > --------------- > * spins_per_delay as extern > * Calculation of skip_wait_list Currently calculation of skip_wait_list is mostly empirical (ie best i measured). I strongly think that instead of spins_per_delay something dependent on concrete lock should be used. I tried to store it in a LWLock itself, but it were worse. Recently I understand it should be stored in array indexed by tranche, but I didn't implement it yet, and therefore didn't measure. [1]: https://commitfest.postgresql.org/14/358/ -- With regards, Sokolov Yura aka funny_falcon Postgres Professional: https://postgrespro.ru The Russian Postgres Company -- Sent via pgsql-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers Attachments: [text/x-diff] lwlock_v3.patch (33.5K, ../../[email protected]/2-lwlock_v3.patch) download | inline diff: From 722a8bed17f82738135264212dde7170b8c0f397 Mon Sep 17 00:00:00 2001 From: Sokolov Yura <[email protected]> Date: Mon, 29 May 2017 09:25:41 +0000 Subject: [PATCH 1/6] More correct use of spinlock inside LWLockWaitListLock. SpinDelayStatus should be function global to count iterations correctly, and produce more correct delays. Also if spin didn't spin, do not count it in spins_per_delay correction. It wasn't necessary before cause delayStatus were used only in contented cases. --- src/backend/storage/lmgr/lwlock.c | 37 +++++++++++++++++-------------------- src/backend/storage/lmgr/s_lock.c | 5 +++++ 2 files changed, 22 insertions(+), 20 deletions(-) diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c index 82a1cf5150..7714085663 100644 --- a/src/backend/storage/lmgr/lwlock.c +++ b/src/backend/storage/lmgr/lwlock.c @@ -323,6 +323,16 @@ get_lwlock_stats_entry(LWLock *lock) } return lwstats; } + +/* just shortcut to not declare lwlock_stats variable at the end of function */ +static void +add_lwlock_stats_spin_stat(LWLock *lock, SpinDelayStatus *delayStatus) +{ + lwlock_stats *lwstats; + + lwstats = get_lwlock_stats_entry(lock); + lwstats->spin_delay_count += delayStatus->delays; +} #endif /* LWLOCK_STATS */ @@ -800,13 +810,9 @@ static void LWLockWaitListLock(LWLock *lock) { uint32 old_state; -#ifdef LWLOCK_STATS - lwlock_stats *lwstats; - uint32 delays = 0; - - lwstats = get_lwlock_stats_entry(lock); -#endif + SpinDelayStatus delayStatus; + init_local_spin_delay(&delayStatus); while (true) { /* always try once to acquire lock directly */ @@ -815,20 +821,10 @@ LWLockWaitListLock(LWLock *lock) break; /* got lock */ /* and then spin without atomic operations until lock is released */ + while (old_state & LW_FLAG_LOCKED) { - SpinDelayStatus delayStatus; - - init_local_spin_delay(&delayStatus); - - while (old_state & LW_FLAG_LOCKED) - { - perform_spin_delay(&delayStatus); - old_state = pg_atomic_read_u32(&lock->state); - } -#ifdef LWLOCK_STATS - delays += delayStatus.delays; -#endif - finish_spin_delay(&delayStatus); + perform_spin_delay(&delayStatus); + old_state = pg_atomic_read_u32(&lock->state); } /* @@ -836,9 +832,10 @@ LWLockWaitListLock(LWLock *lock) * we're attempting to get it again. */ } + finish_spin_delay(&delayStatus); #ifdef LWLOCK_STATS - lwstats->spin_delay_count += delays; + add_lwlock_stats_spin_stat(lock, &delayStatus); #endif } diff --git a/src/backend/storage/lmgr/s_lock.c b/src/backend/storage/lmgr/s_lock.c index 40d8156331..b75c432773 100644 --- a/src/backend/storage/lmgr/s_lock.c +++ b/src/backend/storage/lmgr/s_lock.c @@ -177,6 +177,11 @@ finish_spin_delay(SpinDelayStatus *status) if (status->cur_delay == 0) { /* we never had to delay */ + if (status->spins == 0) + { + /* but we didn't spin either, so ignore */ + return; + } if (spins_per_delay < MAX_SPINS_PER_DELAY) spins_per_delay = Min(spins_per_delay + 100, MAX_SPINS_PER_DELAY); } -- 2.11.0 From 3e13f116daaec1022279d1c7ecd5d5781d8c7527 Mon Sep 17 00:00:00 2001 From: Sokolov Yura <[email protected]> Date: Tue, 30 May 2017 14:05:26 +0300 Subject: [PATCH 2/6] Perform atomic LWLock acquirement or putting into wait list. Before this change, lwlock were acquired in several steps: - first try acquire lockfree (one CAS-spin-loop) - if it were not successful, queue self into wait list (two CAS-spin-loops (in fact, 3 loop, cause of atomic_fetch_and_or)) - try again to acquire lock (another one CAS loop) - if second attempt were successful, deque our self from wait queue (two CAS spin loops). With this change, lock acquired, or wait list lock acquired using single CAS loop: - if it is likely to acquire desired lock mode, we do CAS for that - otherwise we are trying for CAS to lock wait list and set necessary flag (LG_FLAG_HAS_WAITERS) at once. (LWLockWaitListUnlock still performs second CAS loop for releasing wait list lock). Correctness provided by the fact we are trying to lock wait list only when some one holds conflicting lock. If conflicting lock were released, CAS for wait list will not succeed, and we will retry to acquire lock. --- src/backend/storage/lmgr/lwlock.c | 289 ++++++++++++++++++++++---------------- 1 file changed, 171 insertions(+), 118 deletions(-) diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c index 7714085663..b7a9575059 100644 --- a/src/backend/storage/lmgr/lwlock.c +++ b/src/backend/storage/lmgr/lwlock.c @@ -728,20 +728,33 @@ GetLWLockIdentifier(uint32 classId, uint16 eventId) /* * Internal function that tries to atomically acquire the lwlock in the passed - * in mode. + * in mode. If it could not grab the lock, it doesn't puts proc into wait + * queue. * - * This function will not block waiting for a lock to become free - that's the - * callers job. + * It is used only in LWLockConditionalAcquire. * - * Returns true if the lock isn't free and we need to wait. + * Returns true if the lock isn't free. */ static bool -LWLockAttemptLock(LWLock *lock, LWLockMode mode) +LWLockAttemptLockOnce(LWLock *lock, LWLockMode mode) { uint32 old_state; + uint32 mask, + add; AssertArg(mode == LW_EXCLUSIVE || mode == LW_SHARED); + if (mode == LW_EXCLUSIVE) + { + mask = LW_LOCK_MASK; + add = LW_VAL_EXCLUSIVE; + } + else + { + mask = LW_VAL_EXCLUSIVE; + add = LW_VAL_SHARED; + } + /* * Read once outside the loop, later iterations will get the newer value * via compare & exchange. @@ -756,46 +769,126 @@ LWLockAttemptLock(LWLock *lock, LWLockMode mode) desired_state = old_state; - if (mode == LW_EXCLUSIVE) + lock_free = (old_state & mask) == 0; + if (lock_free) { - lock_free = (old_state & LW_LOCK_MASK) == 0; - if (lock_free) - desired_state += LW_VAL_EXCLUSIVE; + desired_state += add; + if (pg_atomic_compare_exchange_u32(&lock->state, + &old_state, desired_state)) + { + /* Great! Got the lock. */ +#ifdef LOCK_DEBUG + if (mode == LW_EXCLUSIVE) + lock->owner = MyProc; +#endif + return false; + } } else { - lock_free = (old_state & LW_VAL_EXCLUSIVE) == 0; - if (lock_free) - desired_state += LW_VAL_SHARED; + /* + * This barrier is never needed for correctness, and it is no-op + * on x86. But probably first iteration of cas loop in + * ProcArrayGroupClearXid will succeed more often with it. + */ + pg_read_barrier(); + return true; } + } + pg_unreachable(); +} - /* - * Attempt to swap in the state we are expecting. If we didn't see - * lock to be free, that's just the old value. If we saw it as free, - * we'll attempt to mark it acquired. The reason that we always swap - * in the value is that this doubles as a memory barrier. We could try - * to be smarter and only swap in values if we saw the lock as free, - * but benchmark haven't shown it as beneficial so far. - * - * Retry if the value changed since we last looked at it. - */ - if (pg_atomic_compare_exchange_u32(&lock->state, - &old_state, desired_state)) +static void LWLockQueueSelfLocked(LWLock *lock, LWLockMode mode); + +/* + * Internal function that tries to atomically acquire the lwlock in the passed + * in mode or put it self into waiting queue with waitmode. + * + * This function will not block waiting for a lock to become free - that's the + * callers job. + * + * Returns true if the lock isn't free and we are in a wait queue. + */ +static inline bool +LWLockAttemptLockOrQueueSelf(LWLock *lock, LWLockMode mode, LWLockMode waitmode) +{ + uint32 old_state; + SpinDelayStatus delayStatus; + bool lock_free; + uint32 mask, + add; + + AssertArg(mode == LW_EXCLUSIVE || mode == LW_SHARED); + + if (mode == LW_EXCLUSIVE) + { + mask = LW_LOCK_MASK; + add = LW_VAL_EXCLUSIVE; + } + else + { + mask = LW_VAL_EXCLUSIVE; + add = LW_VAL_SHARED; + } + + init_local_spin_delay(&delayStatus); + + /* + * Read once outside the loop. Later iterations will get the newer value + * either via compare & exchange or with read after perform_spin_delay. + */ + old_state = pg_atomic_read_u32(&lock->state); + /* loop until we've determined whether we could acquire the lock or not */ + while (true) + { + uint32 desired_state; + + desired_state = old_state; + + lock_free = (old_state & mask) == 0; + if (lock_free) { - if (lock_free) + desired_state += add; + if (pg_atomic_compare_exchange_u32(&lock->state, + &old_state, desired_state)) { /* Great! Got the lock. */ #ifdef LOCK_DEBUG if (mode == LW_EXCLUSIVE) lock->owner = MyProc; #endif - return false; + break; + } + } + else if ((old_state & LW_FLAG_LOCKED) == 0) + { + desired_state |= LW_FLAG_LOCKED | LW_FLAG_HAS_WAITERS; + if (pg_atomic_compare_exchange_u32(&lock->state, + &old_state, desired_state)) + { + LWLockQueueSelfLocked(lock, waitmode); + break; } - else - return true; /* somebody else has the lock */ + } + else + { + perform_spin_delay(&delayStatus); + old_state = pg_atomic_read_u32(&lock->state); } } - pg_unreachable(); +#ifdef LWLOCK_STATS + add_lwlock_stats_spin_stat(lock, &delayStatus); +#endif + + /* + * We intentionally do not call finish_spin_delay here, because the loop + * above usually finished by queuing into the wait list on contention, and + * doesn't reach spins_per_delay thereby doesn't sleep inside of + * perform_spin_delay. Also, different LWLocks has very different + * contention pattern, and it is wrong to update spin-lock statistic based + * on LWLock contention. + */ + return !lock_free; } /* @@ -961,7 +1054,7 @@ LWLockWakeup(LWLock *lock) } /* - * Add ourselves to the end of the queue. + * Add ourselves to the end of the queue, acquiring waitlist lock. * * NB: Mode can be LW_WAIT_UNTIL_FREE here! */ @@ -984,6 +1077,19 @@ LWLockQueueSelf(LWLock *lock, LWLockMode mode) /* setting the flag is protected by the spinlock */ pg_atomic_fetch_or_u32(&lock->state, LW_FLAG_HAS_WAITERS); + LWLockQueueSelfLocked(lock, mode); +} + +/* + * Add ourselves to the end of the queue. + * Lock should be held at this moment, and all neccessary flags are already + * set. It will release wait list lock. + * + * NB: Mode can be LW_WAIT_UNTIL_FREE here! + */ +static void +LWLockQueueSelfLocked(LWLock *lock, LWLockMode mode) +{ MyProc->lwWaiting = true; MyProc->lwWaitMode = mode; @@ -1166,11 +1272,7 @@ LWLockAcquire(LWLock *lock, LWLockMode mode) { bool mustwait; - /* - * Try to grab the lock the first time, we're not in the waitqueue - * yet/anymore. - */ - mustwait = LWLockAttemptLock(lock, mode); + mustwait = LWLockAttemptLockOrQueueSelf(lock, mode, mode); if (!mustwait) { @@ -1179,42 +1281,17 @@ LWLockAcquire(LWLock *lock, LWLockMode mode) } /* - * Ok, at this point we couldn't grab the lock on the first try. We - * cannot simply queue ourselves to the end of the list and wait to be - * woken up because by now the lock could long have been released. - * Instead add us to the queue and try to grab the lock again. If we - * succeed we need to revert the queuing and be happy, otherwise we - * recheck the lock. If we still couldn't grab it, we know that the - * other locker will see our queue entries when releasing since they - * existed before we checked for the lock. - */ - - /* add to the queue */ - LWLockQueueSelf(lock, mode); - - /* we're now guaranteed to be woken up if necessary */ - mustwait = LWLockAttemptLock(lock, mode); - - /* ok, grabbed the lock the second time round, need to undo queueing */ - if (!mustwait) - { - LOG_LWDEBUG("LWLockAcquire", lock, "acquired, undoing queue"); - - LWLockDequeueSelf(lock); - break; - } - - /* * Wait until awakened. * - * Since we share the process wait semaphore with the regular lock - * manager and ProcWaitForSignal, and we may need to acquire an LWLock - * while one of those is pending, it is possible that we get awakened - * for a reason other than being signaled by LWLockRelease. If so, - * loop back and wait again. Once we've gotten the LWLock, - * re-increment the sema by the number of additional signals received, - * so that the lock manager or signal manager will see the received - * signal when it next waits. + * At this point we are already in a wait queue. Since we share the + * process wait semaphore with the regular lock manager and + * ProcWaitForSignal, and we may need to acquire an LWLock while one + * of those is pending, it is possible that we get awakened for a + * reason other than being signaled by LWLockRelease. If so, loop back + * and wait again. Once we've gotten the LWLock, re-increment the + * sema by the number of additional signals received, so that the lock + * manager or signal manager will see the received signal when it next + * waits. */ LOG_LWDEBUG("LWLockAcquire", lock, "waiting"); @@ -1297,7 +1374,7 @@ LWLockConditionalAcquire(LWLock *lock, LWLockMode mode) HOLD_INTERRUPTS(); /* Check for the lock */ - mustwait = LWLockAttemptLock(lock, mode); + mustwait = LWLockAttemptLockOnce(lock, mode); if (mustwait) { @@ -1358,67 +1435,43 @@ LWLockAcquireOrWait(LWLock *lock, LWLockMode mode) */ HOLD_INTERRUPTS(); - /* - * NB: We're using nearly the same twice-in-a-row lock acquisition - * protocol as LWLockAcquire(). Check its comments for details. - */ - mustwait = LWLockAttemptLock(lock, mode); + mustwait = LWLockAttemptLockOrQueueSelf(lock, mode, LW_WAIT_UNTIL_FREE); if (mustwait) { - LWLockQueueSelf(lock, LW_WAIT_UNTIL_FREE); - - mustwait = LWLockAttemptLock(lock, mode); - - if (mustwait) - { - /* - * Wait until awakened. Like in LWLockAcquire, be prepared for - * bogus wakeups, because we share the semaphore with - * ProcWaitForSignal. - */ - LOG_LWDEBUG("LWLockAcquireOrWait", lock, "waiting"); + /* + * Wait until awakened. Like in LWLockAcquire, be prepared for bogus + * wakeups, because we share the semaphore with ProcWaitForSignal. + */ + LOG_LWDEBUG("LWLockAcquireOrWait", lock, "waiting"); #ifdef LWLOCK_STATS - lwstats->block_count++; + lwstats->block_count++; #endif - LWLockReportWaitStart(lock); - TRACE_POSTGRESQL_LWLOCK_WAIT_START(T_NAME(lock), mode); + LWLockReportWaitStart(lock); + TRACE_POSTGRESQL_LWLOCK_WAIT_START(T_NAME(lock), mode); - for (;;) - { - PGSemaphoreLock(proc->sem); - if (!proc->lwWaiting) - break; - extraWaits++; - } + for (;;) + { + PGSemaphoreLock(proc->sem); + if (!proc->lwWaiting) + break; + extraWaits++; + } #ifdef LOCK_DEBUG - { - /* not waiting anymore */ - uint32 nwaiters PG_USED_FOR_ASSERTS_ONLY = pg_atomic_fetch_sub_u32(&lock->nwaiters, 1); - - Assert(nwaiters < MAX_BACKENDS); - } -#endif - TRACE_POSTGRESQL_LWLOCK_WAIT_DONE(T_NAME(lock), mode); - LWLockReportWaitEnd(); - - LOG_LWDEBUG("LWLockAcquireOrWait", lock, "awakened"); - } - else { - LOG_LWDEBUG("LWLockAcquireOrWait", lock, "acquired, undoing queue"); + /* not waiting anymore */ + uint32 nwaiters PG_USED_FOR_ASSERTS_ONLY = pg_atomic_fetch_sub_u32(&lock->nwaiters, 1); - /* - * Got lock in the second attempt, undo queueing. We need to treat - * this as having successfully acquired the lock, otherwise we'd - * not necessarily wake up people we've prevented from acquiring - * the lock. - */ - LWLockDequeueSelf(lock); + Assert(nwaiters < MAX_BACKENDS); } +#endif + TRACE_POSTGRESQL_LWLOCK_WAIT_DONE(T_NAME(lock), mode); + LWLockReportWaitEnd(); + + LOG_LWDEBUG("LWLockAcquireOrWait", lock, "awakened"); } /* -- 2.11.0 From b9a3b25f3a23254a0a3310c763db9215ce4241ac Mon Sep 17 00:00:00 2001 From: Sokolov Yura <[email protected]> Date: Tue, 30 May 2017 17:21:48 +0300 Subject: [PATCH 3/6] use custom loop for LWLockWaitForVar Previously LWLockWaitForVar used following procedure: - check lock state and var value LWLockConflictsWithVar - LWLockQueueSelf - recheck lock state and var with LWLockConflictsWithVar - LWLockDequeueSelf if lock were released by holder or variable changed. With this change, LWLockConflictsWithVar performs single atomic: - check lock state -- if lock is not held, return "no need to wait" -- otherwise try to lock wait list - with lock on wait list check value of variable (return if it was changed) - set flags on a state and queue self into wait list -- if CAS for flags failed, recheck if lock still held. If it is not held, return. With this change there is no need it LWLockQueueSelf and LWLockDequequeSelf any more, so this functions are removed. --- src/backend/storage/lmgr/lwlock.c | 251 +++++++++++--------------------------- 1 file changed, 70 insertions(+), 181 deletions(-) diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c index b7a9575059..83321a89be 100644 --- a/src/backend/storage/lmgr/lwlock.c +++ b/src/backend/storage/lmgr/lwlock.c @@ -176,7 +176,6 @@ typedef struct lwlock_stats int sh_acquire_count; int ex_acquire_count; int block_count; - int dequeue_self_count; int spin_delay_count; } lwlock_stats; @@ -284,11 +283,11 @@ print_lwlock_stats(int code, Datum arg) while ((lwstats = (lwlock_stats *) hash_seq_search(&scan)) != NULL) { fprintf(stderr, - "PID %d lwlock %s %p: shacq %u exacq %u blk %u spindelay %u dequeue self %u\n", + "PID %d lwlock %s %p: shacq %u exacq %u blk %u spindelay %u\n", MyProcPid, LWLockTrancheArray[lwstats->key.tranche], lwstats->key.instance, lwstats->sh_acquire_count, lwstats->ex_acquire_count, lwstats->block_count, - lwstats->spin_delay_count, lwstats->dequeue_self_count); + lwstats->spin_delay_count); } LWLockRelease(&MainLWLockArray[0].lock); @@ -318,7 +317,6 @@ get_lwlock_stats_entry(LWLock *lock) lwstats->sh_acquire_count = 0; lwstats->ex_acquire_count = 0; lwstats->block_count = 0; - lwstats->dequeue_self_count = 0; lwstats->spin_delay_count = 0; } return lwstats; @@ -1054,33 +1052,6 @@ LWLockWakeup(LWLock *lock) } /* - * Add ourselves to the end of the queue, acquiring waitlist lock. - * - * NB: Mode can be LW_WAIT_UNTIL_FREE here! - */ -static void -LWLockQueueSelf(LWLock *lock, LWLockMode mode) -{ - /* - * If we don't have a PGPROC structure, there's no way to wait. This - * should never occur, since MyProc should only be null during shared - * memory initialization. - */ - if (MyProc == NULL) - elog(PANIC, "cannot wait without a PGPROC structure"); - - if (MyProc->lwWaiting) - elog(PANIC, "queueing for lock while waiting on another one"); - - LWLockWaitListLock(lock); - - /* setting the flag is protected by the spinlock */ - pg_atomic_fetch_or_u32(&lock->state, LW_FLAG_HAS_WAITERS); - - LWLockQueueSelfLocked(lock, mode); -} - -/* * Add ourselves to the end of the queue. * Lock should be held at this moment, and all neccessary flags are already * set. It will release wait list lock. @@ -1109,100 +1080,6 @@ LWLockQueueSelfLocked(LWLock *lock, LWLockMode mode) } /* - * Remove ourselves from the waitlist. - * - * This is used if we queued ourselves because we thought we needed to sleep - * but, after further checking, we discovered that we don't actually need to - * do so. - */ -static void -LWLockDequeueSelf(LWLock *lock) -{ - bool found = false; - proclist_mutable_iter iter; - -#ifdef LWLOCK_STATS - lwlock_stats *lwstats; - - lwstats = get_lwlock_stats_entry(lock); - - lwstats->dequeue_self_count++; -#endif - - LWLockWaitListLock(lock); - - /* - * Can't just remove ourselves from the list, but we need to iterate over - * all entries as somebody else could have dequeued us. - */ - proclist_foreach_modify(iter, &lock->waiters, lwWaitLink) - { - if (iter.cur == MyProc->pgprocno) - { - found = true; - proclist_delete(&lock->waiters, iter.cur, lwWaitLink); - break; - } - } - - if (proclist_is_empty(&lock->waiters) && - (pg_atomic_read_u32(&lock->state) & LW_FLAG_HAS_WAITERS) != 0) - { - pg_atomic_fetch_and_u32(&lock->state, ~LW_FLAG_HAS_WAITERS); - } - - /* XXX: combine with fetch_and above? */ - LWLockWaitListUnlock(lock); - - /* clear waiting state again, nice for debugging */ - if (found) - MyProc->lwWaiting = false; - else - { - int extraWaits = 0; - - /* - * Somebody else dequeued us and has or will wake us up. Deal with the - * superfluous absorption of a wakeup. - */ - - /* - * Reset releaseOk if somebody woke us before we removed ourselves - - * they'll have set it to false. - */ - pg_atomic_fetch_or_u32(&lock->state, LW_FLAG_RELEASE_OK); - - /* - * Now wait for the scheduled wakeup, otherwise our ->lwWaiting would - * get reset at some inconvenient point later. Most of the time this - * will immediately return. - */ - for (;;) - { - PGSemaphoreLock(MyProc->sem); - if (!MyProc->lwWaiting) - break; - extraWaits++; - } - - /* - * Fix the process wait semaphore's count for any absorbed wakeups. - */ - while (extraWaits-- > 0) - PGSemaphoreUnlock(MyProc->sem); - } - -#ifdef LOCK_DEBUG - { - /* not waiting anymore */ - uint32 nwaiters PG_USED_FOR_ASSERTS_ONLY = pg_atomic_fetch_sub_u32(&lock->nwaiters, 1); - - Assert(nwaiters < MAX_BACKENDS); - } -#endif -} - -/* * LWLockAcquire - acquire a lightweight lock in the specified mode * * If the lock is not available, sleep until it is. Returns true if the lock @@ -1515,43 +1392,87 @@ LWLockConflictsWithVar(LWLock *lock, { bool mustwait; uint64 value; + uint32 state; + SpinDelayStatus delayStatus; - /* - * Test first to see if it the slot is free right now. - * - * XXX: the caller uses a spinlock before this, so we don't need a memory - * barrier here as far as the current usage is concerned. But that might - * not be safe in general. - */ - mustwait = (pg_atomic_read_u32(&lock->state) & LW_VAL_EXCLUSIVE) != 0; - - if (!mustwait) - { - *result = true; - return false; - } - - *result = false; + init_local_spin_delay(&delayStatus); /* + * We are trying to detect exclusive lock on state, or value change. If + * neither happens, we put ourself into wait queue. + * * Read value using the lwlock's wait list lock, as we can't generally * rely on atomic 64 bit reads/stores. TODO: On platforms with a way to * do atomic 64 bit reads/writes the spinlock should be optimized away. */ - LWLockWaitListLock(lock); - value = *valptr; - LWLockWaitListUnlock(lock); + while (true) + { + state = pg_atomic_read_u32(&lock->state); + mustwait = (state & LW_VAL_EXCLUSIVE) != 0; - if (value != oldval) + if (!mustwait) + { + *result = true; + goto exit; + } + + if ((state & LW_FLAG_LOCKED) == 0) + { + if (pg_atomic_compare_exchange_u32(&lock->state, &state, + state | LW_FLAG_LOCKED)) + break; + } + perform_spin_delay(&delayStatus); + } + + /* Now we've locked wait queue, and someone holds EXCLUSIVE lock. */ + value = *valptr; + mustwait = value == oldval; + if (!mustwait) { - mustwait = false; + /* value changed, no need to block */ + LWLockWaitListUnlock(lock); *newval = value; + goto exit; } - else + + /* + * If value didn't change, we have to set LW_FLAG_HAS_WAITERS and + * LW_FLAG_RELEASE_OK flags and put ourself into wait queue. We could find + * that the lock is not held anymore, in this case we should return + * without blocking. + * + * Note: value could not change again cause we are holding WaitList lock. + */ + while (true) { - mustwait = true; - } + uint32 desired_state; + desired_state = state | LW_FLAG_HAS_WAITERS | LW_FLAG_RELEASE_OK; + if (pg_atomic_compare_exchange_u32(&lock->state, &state, + desired_state)) + { + /* + * we satisfy invariant: flags should be set while someone holds + * the lock + */ + LWLockQueueSelfLocked(lock, LW_WAIT_UNTIL_FREE); + break; + } + if ((state & LW_VAL_EXCLUSIVE) == 0) + { + /* lock was released, no need to block anymore */ + LWLockWaitListUnlock(lock); + *result = true; + mustwait = false; + break; + } + } +exit: +#ifdef LWLOCK_STATS + add_lwlock_stats_spin_stat(lock, &delayStatus); +#endif + finish_spin_delay(&delayStatus); return mustwait; } @@ -1603,38 +1524,6 @@ LWLockWaitForVar(LWLock *lock, uint64 *valptr, uint64 oldval, uint64 *newval) break; /* the lock was free or value didn't match */ /* - * Add myself to wait queue. Note that this is racy, somebody else - * could wakeup before we're finished queuing. NB: We're using nearly - * the same twice-in-a-row lock acquisition protocol as - * LWLockAcquire(). Check its comments for details. The only - * difference is that we also have to check the variable's values when - * checking the state of the lock. - */ - LWLockQueueSelf(lock, LW_WAIT_UNTIL_FREE); - - /* - * Set RELEASE_OK flag, to make sure we get woken up as soon as the - * lock is released. - */ - pg_atomic_fetch_or_u32(&lock->state, LW_FLAG_RELEASE_OK); - - /* - * We're now guaranteed to be woken up if necessary. Recheck the lock - * and variables state. - */ - mustwait = LWLockConflictsWithVar(lock, valptr, oldval, newval, - &result); - - /* Ok, no conflict after we queued ourselves. Undo queueing. */ - if (!mustwait) - { - LOG_LWDEBUG("LWLockWaitForVar", lock, "free, undoing queue"); - - LWLockDequeueSelf(lock); - break; - } - - /* * Wait until awakened. * * Since we share the process wait semaphore with the regular lock -- 2.11.0 From adc1a5061cd8d7baa9337971acdc0b14f839c0b8 Mon Sep 17 00:00:00 2001 From: Sokolov Yura <[email protected]> Date: Wed, 31 May 2017 13:56:30 +0000 Subject: [PATCH 4/6] Make acquiring LWLock to look more like spinlock. If delayStatus.spins <= spin_per_delay/8, then do not try to add itself into wait list, but instead do perform_spin_delay and retry lock acquiring. Since delayStatus.spins is reset after each pg_usleep, it spins after sleep again. It greatly improves performance on "contended but not much" cases, and overall performance also. --- src/backend/storage/lmgr/lwlock.c | 10 ++++++++-- src/backend/storage/lmgr/s_lock.c | 2 +- src/include/storage/s_lock.h | 2 ++ 3 files changed, 11 insertions(+), 3 deletions(-) diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c index 83321a89be..334c2a2d96 100644 --- a/src/backend/storage/lmgr/lwlock.c +++ b/src/backend/storage/lmgr/lwlock.c @@ -815,6 +815,7 @@ LWLockAttemptLockOrQueueSelf(LWLock *lock, LWLockMode mode, LWLockMode waitmode) bool lock_free; uint32 mask, add; + int skip_wait_list; AssertArg(mode == LW_EXCLUSIVE || mode == LW_SHARED); @@ -822,11 +823,13 @@ LWLockAttemptLockOrQueueSelf(LWLock *lock, LWLockMode mode, LWLockMode waitmode) { mask = LW_LOCK_MASK; add = LW_VAL_EXCLUSIVE; + skip_wait_list = spins_per_delay / 8; } else { mask = LW_VAL_EXCLUSIVE; add = LW_VAL_SHARED; + skip_wait_list = spins_per_delay / 4; } init_local_spin_delay(&delayStatus); @@ -858,7 +861,7 @@ LWLockAttemptLockOrQueueSelf(LWLock *lock, LWLockMode mode, LWLockMode waitmode) break; } } - else if ((old_state & LW_FLAG_LOCKED) == 0) + else if (delayStatus.spins > skip_wait_list && (old_state & LW_FLAG_LOCKED) == 0) { desired_state |= LW_FLAG_LOCKED | LW_FLAG_HAS_WAITERS; if (pg_atomic_compare_exchange_u32(&lock->state, @@ -1394,9 +1397,12 @@ LWLockConflictsWithVar(LWLock *lock, uint64 value; uint32 state; SpinDelayStatus delayStatus; + int skip_wait_list; init_local_spin_delay(&delayStatus); + skip_wait_list = spins_per_delay / 4; + /* * We are trying to detect exclusive lock on state, or value change. If * neither happens, we put ourself into wait queue. @@ -1416,7 +1422,7 @@ LWLockConflictsWithVar(LWLock *lock, goto exit; } - if ((state & LW_FLAG_LOCKED) == 0) + if (delayStatus.spins > skip_wait_list && (state & LW_FLAG_LOCKED) == 0) { if (pg_atomic_compare_exchange_u32(&lock->state, &state, state | LW_FLAG_LOCKED)) diff --git a/src/backend/storage/lmgr/s_lock.c b/src/backend/storage/lmgr/s_lock.c index b75c432773..0ba3bd19d8 100644 --- a/src/backend/storage/lmgr/s_lock.c +++ b/src/backend/storage/lmgr/s_lock.c @@ -63,7 +63,7 @@ slock_t dummy_spinlock; -static int spins_per_delay = DEFAULT_SPINS_PER_DELAY; +int spins_per_delay = DEFAULT_SPINS_PER_DELAY; /* diff --git a/src/include/storage/s_lock.h b/src/include/storage/s_lock.h index bbf505e246..1dd4c7880d 100644 --- a/src/include/storage/s_lock.h +++ b/src/include/storage/s_lock.h @@ -964,6 +964,8 @@ extern int s_lock(volatile slock_t *lock, const char *file, int line, const char /* Support for dynamic adjustment of spins_per_delay */ #define DEFAULT_SPINS_PER_DELAY 100 +/* Read in lwlock.c to adjust behavior */ +extern int spins_per_delay; extern void set_spins_per_delay(int shared_spins_per_delay); extern int update_spins_per_delay(int shared_spins_per_delay); -- 2.11.0 From e5b13550fc48d62b0b855bedd7fcd5848b806b09 Mon Sep 17 00:00:00 2001 From: Sokolov Yura <[email protected]> Date: Tue, 30 May 2017 18:54:25 +0300 Subject: [PATCH 5/6] Fix implementation description in a lwlock.c . --- src/backend/storage/lmgr/lwlock.c | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c index 334c2a2d96..0a41c2c4e2 100644 --- a/src/backend/storage/lmgr/lwlock.c +++ b/src/backend/storage/lmgr/lwlock.c @@ -62,16 +62,15 @@ * notice that we have to wait. Unfortunately by the time we have finished * queuing, the former locker very well might have already finished it's * work. That's problematic because we're now stuck waiting inside the OS. - - * To mitigate those races we use a two phased attempt at locking: - * Phase 1: Try to do it atomically, if we succeed, nice - * Phase 2: Add ourselves to the waitqueue of the lock - * Phase 3: Try to grab the lock again, if we succeed, remove ourselves from - * the queue - * Phase 4: Sleep till wake-up, goto Phase 1 * - * This protects us against the problem from above as nobody can release too - * quick, before we're queued, since after Phase 2 we're already queued. + * This race is avoided by taking a lock for the wait list using CAS with the old + * state value, so it would only succeed if lock is still held. Necessary flag + * is set using the same CAS. + * + * Inside LWLockConflictsWithVar the wait list lock is reused to protect the variable + * value. First the lock is acquired to check the variable value, then flags are + * set with a second CAS before queuing to the wait list in order to ensure that the lock was not + * released yet. * ------------------------------------------------------------------------- */ #include "postgres.h" -- 2.11.0 From cc74a849a64e331930a2285e15445d7f08b54169 Mon Sep 17 00:00:00 2001 From: Sokolov Yura <[email protected]> Date: Fri, 2 Jun 2017 11:34:23 +0000 Subject: [PATCH 6/6] Make SpinDelayStatus a bit lighter. It saves couple of instruction of fast path of spin-loop, and so makes fast path of LWLock a bit faster (and in other places where spinlock is used). Also it makes call to perform_spin_delay a bit slower, that could positively affect on spin behavior, especially if no `pause` instruction present. --- src/backend/storage/lmgr/s_lock.c | 9 +++++---- src/include/storage/s_lock.h | 16 ++++++---------- 2 files changed, 11 insertions(+), 14 deletions(-) diff --git a/src/backend/storage/lmgr/s_lock.c b/src/backend/storage/lmgr/s_lock.c index 0ba3bd19d8..2367014031 100644 --- a/src/backend/storage/lmgr/s_lock.c +++ b/src/backend/storage/lmgr/s_lock.c @@ -93,11 +93,11 @@ s_lock(volatile slock_t *lock, const char *file, int line, const char *func) { SpinDelayStatus delayStatus; - init_spin_delay(&delayStatus, file, line, func); + init_spin_delay(&delayStatus); while (TAS_SPIN(lock)) { - perform_spin_delay(&delayStatus); + perform_spin_delay_with_info(&delayStatus, file, line, func); } finish_spin_delay(&delayStatus); @@ -122,7 +122,8 @@ s_unlock(volatile slock_t *lock) * Wait while spinning on a contended spinlock. */ void -perform_spin_delay(SpinDelayStatus *status) +perform_spin_delay_with_info(SpinDelayStatus *status, + const char *file, int line, const char *func) { /* CPU-specific delay each time through the loop */ SPIN_DELAY(); @@ -131,7 +132,7 @@ perform_spin_delay(SpinDelayStatus *status) if (++(status->spins) >= spins_per_delay) { if (++(status->delays) > NUM_DELAYS) - s_lock_stuck(status->file, status->line, status->func); + s_lock_stuck(file, line, func); if (status->cur_delay == 0) /* first time to delay? */ status->cur_delay = MIN_DELAY_USEC; diff --git a/src/include/storage/s_lock.h b/src/include/storage/s_lock.h index 1dd4c7880d..e5c4645461 100644 --- a/src/include/storage/s_lock.h +++ b/src/include/storage/s_lock.h @@ -979,25 +979,21 @@ typedef struct int spins; int delays; int cur_delay; - const char *file; - int line; - const char *func; } SpinDelayStatus; static inline void -init_spin_delay(SpinDelayStatus *status, - const char *file, int line, const char *func) +init_spin_delay(SpinDelayStatus *status) { status->spins = 0; status->delays = 0; status->cur_delay = 0; - status->file = file; - status->line = line; - status->func = func; } -#define init_local_spin_delay(status) init_spin_delay(status, __FILE__, __LINE__, PG_FUNCNAME_MACRO) -void perform_spin_delay(SpinDelayStatus *status); +#define init_local_spin_delay(status) init_spin_delay(status) +void perform_spin_delay_with_info(SpinDelayStatus *status, + const char *file, int line, const char *func); +#define perform_spin_delay(status) \ + perform_spin_delay_with_info(status, __FILE__, __LINE__, PG_FUNCNAME_MACRO) void finish_spin_delay(SpinDelayStatus *status); #endif /* S_LOCK_H */ -- 2.11.0 ^ permalink raw reply [nested|flat] 62+ messages in thread
* Re: Fix performance degradation of contended LWLock on NUMA @ 2017-09-11 15:01 Jesper Pedersen <[email protected]> parent: Sokolov Yura <[email protected]> 1 sibling, 1 reply; 62+ messages in thread From: Jesper Pedersen @ 2017-09-11 15:01 UTC (permalink / raw) To: Sokolov Yura <[email protected]>; +Cc: pgsql-hackers; Andres Freund <[email protected]> Hi, On 09/08/2017 03:35 PM, Sokolov Yura wrote: >> I'm seeing >> >> -M prepared: Up to 11% improvement >> -M prepared -S: No improvement, no regression ("noise") >> -M prepared -N: Up to 12% improvement >> >> for all runs the improvement shows up the closer you get to the number >> of CPU threads, or above. Although I'm not seeing the same >> improvements as you on very large client counts there are definitely >> improvements :) > > It is expected: > - patch "fixes NUMA": for example, it doesn't give improvement on 1 socket > at all (I've tested it using numactl to bind to 1 socket) > - and certainly it gives less improvement on 2 sockets than on 4 sockets > (and 28 cores vs 72 cores also gives difference), > - one of hot points were CLogControlLock, and it were fixed with > "Group mode for CLOG updates" [1] > I'm planning to re-test that patch. >> >> +static inline bool >> +LWLockAttemptLockOrQueueSelf(LWLock *lock, LWLockMode mode, >> LWLockMode waitmode) >> >> I'll leave it to the Committer to decide if this method is too big to >> be "inline". > > GCC 4.9 doesn't want to inline it without directive, and function call > is then remarkable in profile. > > Attach contains version with all suggestions applied except remove of > "inline". > Yes, ideally the method will be kept at "inline". >> Open questions: >> --------------- >> * spins_per_delay as extern >> * Calculation of skip_wait_list > > Currently calculation of skip_wait_list is mostly empirical (ie best > i measured). > Ok, good to know. > I strongly think that instead of spins_per_delay something dependent > on concrete lock should be used. I tried to store it in a LWLock > itself, but it were worse. Yes, LWLock should be kept as small as possible, and cache line aligned due to the cache storms, as shown by perf c2c. > Recently I understand it should be stored in array indexed by tranche, > but I didn't implement it yet, and therefore didn't measure. > Different constants for the LWLock could have an impact, but the constants would also be dependent on machine setup, and work load. Thanks for working on this ! Best regards, Jesper -- Sent via pgsql-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers ^ permalink raw reply [nested|flat] 62+ messages in thread
* Re: Fix performance degradation of contended LWLock on NUMA @ 2017-09-14 12:34 Jesper Pedersen <[email protected]> parent: Jesper Pedersen <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Jesper Pedersen @ 2017-09-14 12:34 UTC (permalink / raw) To: Sokolov Yura <[email protected]>; +Cc: pgsql-hackers; Andres Freund <[email protected]> On 09/11/2017 11:01 AM, Jesper Pedersen wrote: > Thanks for working on this ! > Moved to "Ready for Committer". Best regards, Jesper -- Sent via pgsql-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers ^ permalink raw reply [nested|flat] 62+ messages in thread
* Re: Fix performance degradation of contended LWLock on NUMA @ 2017-10-19 00:03 Andres Freund <[email protected]> parent: Sokolov Yura <[email protected]> 1 sibling, 1 reply; 62+ messages in thread From: Andres Freund @ 2017-10-19 00:03 UTC (permalink / raw) To: Sokolov Yura <[email protected]>; +Cc: [email protected]; pgsql-hackers; [email protected] Hi, On 2017-09-08 22:35:39 +0300, Sokolov Yura wrote: > From 722a8bed17f82738135264212dde7170b8c0f397 Mon Sep 17 00:00:00 2001 > From: Sokolov Yura <[email protected]> > Date: Mon, 29 May 2017 09:25:41 +0000 > Subject: [PATCH 1/6] More correct use of spinlock inside LWLockWaitListLock. > > SpinDelayStatus should be function global to count iterations correctly, > and produce more correct delays. > > Also if spin didn't spin, do not count it in spins_per_delay correction. > It wasn't necessary before cause delayStatus were used only in contented > cases. I'm not convinced this is benefial. Adds a bit of overhead to the casewhere LW_FLAG_LOCKED can be set immediately. OTOH, it's not super likely to make a large difference if the lock has to be taken anyway... > + > +/* just shortcut to not declare lwlock_stats variable at the end of function */ > +static void > +add_lwlock_stats_spin_stat(LWLock *lock, SpinDelayStatus *delayStatus) > +{ > + lwlock_stats *lwstats; > + > + lwstats = get_lwlock_stats_entry(lock); > + lwstats->spin_delay_count += delayStatus->delays; > +} > #endif /* LWLOCK_STATS */ This seems like a pretty independent change. > /* > * Internal function that tries to atomically acquire the lwlock in the passed > - * in mode. > + * in mode. If it could not grab the lock, it doesn't puts proc into wait > + * queue. > * > - * This function will not block waiting for a lock to become free - that's the > - * callers job. > + * It is used only in LWLockConditionalAcquire. > * > - * Returns true if the lock isn't free and we need to wait. > + * Returns true if the lock isn't free. > */ > static bool > -LWLockAttemptLock(LWLock *lock, LWLockMode mode) > +LWLockAttemptLockOnce(LWLock *lock, LWLockMode mode) This now has become a fairly special cased function, I'm not convinced it makes much sense with the current naming and functionality. > +/* > + * Internal function that tries to atomically acquire the lwlock in the passed > + * in mode or put it self into waiting queue with waitmode. > + * This function will not block waiting for a lock to become free - that's the > + * callers job. > + * > + * Returns true if the lock isn't free and we are in a wait queue. > + */ > +static inline bool > +LWLockAttemptLockOrQueueSelf(LWLock *lock, LWLockMode mode, LWLockMode waitmode) > +{ > + uint32 old_state; > + SpinDelayStatus delayStatus; > + bool lock_free; > + uint32 mask, > + add; > + > + AssertArg(mode == LW_EXCLUSIVE || mode == LW_SHARED); > + > + if (mode == LW_EXCLUSIVE) > + { > + mask = LW_LOCK_MASK; > + add = LW_VAL_EXCLUSIVE; > + } > + else > + { > + mask = LW_VAL_EXCLUSIVE; > + add = LW_VAL_SHARED; > + } > + > + init_local_spin_delay(&delayStatus); The way you moved this around has the disadvantage that we now do this - a number of writes - even in the very common case where the lwlock can be acquired directly. > + /* > + * Read once outside the loop. Later iterations will get the newer value > + * either via compare & exchange or with read after perform_spin_delay. > + */ > + old_state = pg_atomic_read_u32(&lock->state); > + /* loop until we've determined whether we could acquire the lock or not */ > + while (true) > + { > + uint32 desired_state; > + > + desired_state = old_state; > + > + lock_free = (old_state & mask) == 0; > + if (lock_free) > { > - if (lock_free) > + desired_state += add; > + if (pg_atomic_compare_exchange_u32(&lock->state, > + &old_state, desired_state)) > { > /* Great! Got the lock. */ > #ifdef LOCK_DEBUG > if (mode == LW_EXCLUSIVE) > lock->owner = MyProc; > #endif > - return false; > + break; > + } > + } > + else if ((old_state & LW_FLAG_LOCKED) == 0) > + { > + desired_state |= LW_FLAG_LOCKED | LW_FLAG_HAS_WAITERS; > + if (pg_atomic_compare_exchange_u32(&lock->state, > + &old_state, desired_state)) > + { > + LWLockQueueSelfLocked(lock, waitmode); > + break; > } > - else > - return true; /* somebody else has the lock */ > + } > + else > + { > + perform_spin_delay(&delayStatus); > + old_state = pg_atomic_read_u32(&lock->state); > } > } > - pg_unreachable(); > +#ifdef LWLOCK_STATS > + add_lwlock_stats_spin_stat(lock, &delayStatus); > +#endif > + > + /* > + * We intentionally do not call finish_spin_delay here, because the loop > + * above usually finished by queuing into the wait list on contention, and > + * doesn't reach spins_per_delay thereby doesn't sleep inside of > + * perform_spin_delay. Also, different LWLocks has very different > + * contention pattern, and it is wrong to update spin-lock statistic based > + * on LWLock contention. > + */ Huh? This seems entirely unconvincing. Without adjusting this here we'll just spin the same way every iteration. Especially for the case where somebody else holds LW_FLAG_LOCKED that's quite bad. > From e5b13550fc48d62b0b855bedd7fcd5848b806b09 Mon Sep 17 00:00:00 2001 > From: Sokolov Yura <[email protected]> > Date: Tue, 30 May 2017 18:54:25 +0300 > Subject: [PATCH 5/6] Fix implementation description in a lwlock.c . > > --- > src/backend/storage/lmgr/lwlock.c | 17 ++++++++--------- > 1 file changed, 8 insertions(+), 9 deletions(-) > > diff --git a/src/backend/storage/lmgr/lwlock.c b/src/backend/storage/lmgr/lwlock.c > index 334c2a2d96..0a41c2c4e2 100644 > --- a/src/backend/storage/lmgr/lwlock.c > +++ b/src/backend/storage/lmgr/lwlock.c > @@ -62,16 +62,15 @@ > * notice that we have to wait. Unfortunately by the time we have finished > * queuing, the former locker very well might have already finished it's > * work. That's problematic because we're now stuck waiting inside the OS. > - > - * To mitigate those races we use a two phased attempt at locking: > - * Phase 1: Try to do it atomically, if we succeed, nice > - * Phase 2: Add ourselves to the waitqueue of the lock > - * Phase 3: Try to grab the lock again, if we succeed, remove ourselves from > - * the queue > - * Phase 4: Sleep till wake-up, goto Phase 1 > * > - * This protects us against the problem from above as nobody can release too > - * quick, before we're queued, since after Phase 2 we're already queued. > + * This race is avoided by taking a lock for the wait list using CAS with the old > + * state value, so it would only succeed if lock is still held. Necessary flag > + * is set using the same CAS. > + * > + * Inside LWLockConflictsWithVar the wait list lock is reused to protect the variable > + * value. First the lock is acquired to check the variable value, then flags are > + * set with a second CAS before queuing to the wait list in order to ensure that the lock was not > + * released yet. > * ------------------------------------------------------------------------- > */ I think this needs more extensive surgery. > From cc74a849a64e331930a2285e15445d7f08b54169 Mon Sep 17 00:00:00 2001 > From: Sokolov Yura <[email protected]> > Date: Fri, 2 Jun 2017 11:34:23 +0000 > Subject: [PATCH 6/6] Make SpinDelayStatus a bit lighter. > > It saves couple of instruction of fast path of spin-loop, and so makes > fast path of LWLock a bit faster (and in other places where spinlock is > used). > Also it makes call to perform_spin_delay a bit slower, that could > positively affect on spin behavior, especially if no `pause` instruction > present. Whaa? That seems pretty absurd reasoning. One big advantage of this is that we should be able to make LW_SHARED acquisition an xadd. I'd done that previously, when the separate spinlock still existed, and it was quite beneficial for performance on larger systems. But it was some fiddly code - should be easier now. Requires some careful management when noticing that an xadd acquired a shared lock even though there's a conflicting exclusive lock, but increase the fastpath. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers ^ permalink raw reply [nested|flat] 62+ messages in thread
* Re: Fix performance degradation of contended LWLock on NUMA @ 2017-10-19 11:36 Sokolov Yura <[email protected]> parent: Andres Freund <[email protected]> 0 siblings, 2 replies; 62+ messages in thread From: Sokolov Yura @ 2017-10-19 11:36 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: [email protected]; pgsql-hackers Hi, On 2017-10-19 03:03, Andres Freund wrote: > Hi, > > On 2017-09-08 22:35:39 +0300, Sokolov Yura wrote: >> /* >> * Internal function that tries to atomically acquire the lwlock in >> the passed >> - * in mode. >> + * in mode. If it could not grab the lock, it doesn't puts proc into >> wait >> + * queue. >> * >> - * This function will not block waiting for a lock to become free - >> that's the >> - * callers job. >> + * It is used only in LWLockConditionalAcquire. >> * >> - * Returns true if the lock isn't free and we need to wait. >> + * Returns true if the lock isn't free. >> */ >> static bool >> -LWLockAttemptLock(LWLock *lock, LWLockMode mode) >> +LWLockAttemptLockOnce(LWLock *lock, LWLockMode mode) > > This now has become a fairly special cased function, I'm not convinced > it makes much sense with the current naming and functionality. It just had not to be as complex as LWLockAttpemptLockOrQueueSelf. Their functionality is too different. >> +/* >> + * Internal function that tries to atomically acquire the lwlock in >> the passed >> + * in mode or put it self into waiting queue with waitmode. >> + * This function will not block waiting for a lock to become free - >> that's the >> + * callers job. >> + * >> + * Returns true if the lock isn't free and we are in a wait queue. >> + */ >> +static inline bool >> +LWLockAttemptLockOrQueueSelf(LWLock *lock, LWLockMode mode, >> LWLockMode waitmode) >> +{ >> + uint32 old_state; >> + SpinDelayStatus delayStatus; >> + bool lock_free; >> + uint32 mask, >> + add; >> + >> + AssertArg(mode == LW_EXCLUSIVE || mode == LW_SHARED); >> + >> + if (mode == LW_EXCLUSIVE) >> + { >> + mask = LW_LOCK_MASK; >> + add = LW_VAL_EXCLUSIVE; >> + } >> + else >> + { >> + mask = LW_VAL_EXCLUSIVE; >> + add = LW_VAL_SHARED; >> + } >> + >> + init_local_spin_delay(&delayStatus); > > The way you moved this around has the disadvantage that we now do this > - > a number of writes - even in the very common case where the lwlock can > be acquired directly. Excuse me, I don't understand fine. Do you complain against init_local_spin_delay placed here? Placing it in other place will complicate code. Or you complain against setting `mask` and `add`? At least it simplifies loop body. For example, it is simpler to write specialized Power assembly using already filled `mask+add` registers (i have a working draft with power assembly). In both cases, I think simpler version should be accepted first. It acts as algorithm definition. And it already gives measurable improvement. After some testing, attempt to improve it may be performed (assuming release will be in a next autumn, there is enough time for testing and improvements) I can be mistaken. >> + /* >> + * We intentionally do not call finish_spin_delay here, because the >> loop >> + * above usually finished by queuing into the wait list on >> contention, and >> + * doesn't reach spins_per_delay thereby doesn't sleep inside of >> + * perform_spin_delay. Also, different LWLocks has very different >> + * contention pattern, and it is wrong to update spin-lock statistic >> based >> + * on LWLock contention. >> + */ > > Huh? This seems entirely unconvincing. Without adjusting this here > we'll > just spin the same way every iteration. Especially for the case where > somebody else holds LW_FLAG_LOCKED that's quite bad. LWLock's are very different. Some of them are always short-term (BufferLock), others are always locked for a long time. Some sparse, and some crowded. It is quite bad to tune single variable `spins_per_delay` by such different load. And it is already tuned by spin-locks. And spin-locks tune it quite well. I've tried to place this delay into lock itself (it has 2 free bytes), but this attempt performed worse. Now I understand, that delays should be stored in array indexed by tranche. But I have no time to test this idea. And I doubt it will give cardinally better results (ie > 5%), so I think it is better to accept patch in this way, and then experiment with per-tranche delay. > >> From e5b13550fc48d62b0b855bedd7fcd5848b806b09 Mon Sep 17 00:00:00 2001 >> From: Sokolov Yura <[email protected]> >> Date: Tue, 30 May 2017 18:54:25 +0300 >> Subject: [PATCH 5/6] Fix implementation description in a lwlock.c . >> >> --- >> src/backend/storage/lmgr/lwlock.c | 17 ++++++++--------- >> 1 file changed, 8 insertions(+), 9 deletions(-) >> >> diff --git a/src/backend/storage/lmgr/lwlock.c >> b/src/backend/storage/lmgr/lwlock.c >> index 334c2a2d96..0a41c2c4e2 100644 >> --- a/src/backend/storage/lmgr/lwlock.c >> +++ b/src/backend/storage/lmgr/lwlock.c >> @@ -62,16 +62,15 @@ >> * notice that we have to wait. Unfortunately by the time we have >> finished >> * queuing, the former locker very well might have already finished >> it's >> * work. That's problematic because we're now stuck waiting inside >> the OS. >> - >> - * To mitigate those races we use a two phased attempt at locking: >> - * Phase 1: Try to do it atomically, if we succeed, nice >> - * Phase 2: Add ourselves to the waitqueue of the lock >> - * Phase 3: Try to grab the lock again, if we succeed, remove >> ourselves from >> - * the queue >> - * Phase 4: Sleep till wake-up, goto Phase 1 >> * >> - * This protects us against the problem from above as nobody can >> release too >> - * quick, before we're queued, since after Phase 2 we're already >> queued. >> + * This race is avoided by taking a lock for the wait list using CAS >> with the old >> + * state value, so it would only succeed if lock is still held. >> Necessary flag >> + * is set using the same CAS. >> + * >> + * Inside LWLockConflictsWithVar the wait list lock is reused to >> protect the variable >> + * value. First the lock is acquired to check the variable value, >> then flags are >> + * set with a second CAS before queuing to the wait list in order to >> ensure that the lock was not >> + * released yet. >> * >> ------------------------------------------------------------------------- >> */ > > I think this needs more extensive surgery. Again I don't understand what you mean. Looks like my English is not very fluent :-( Do you mean, there should be more thorough description? Is description from letter is good enough to be copied into this comment? >> From cc74a849a64e331930a2285e15445d7f08b54169 Mon Sep 17 00:00:00 2001 >> From: Sokolov Yura <[email protected]> >> Date: Fri, 2 Jun 2017 11:34:23 +0000 >> Subject: [PATCH 6/6] Make SpinDelayStatus a bit lighter. >> >> It saves couple of instruction of fast path of spin-loop, and so makes >> fast path of LWLock a bit faster (and in other places where spinlock >> is >> used). > >> Also it makes call to perform_spin_delay a bit slower, that could >> positively affect on spin behavior, especially if no `pause` >> instruction >> present. > > Whaa? That seems pretty absurd reasoning. :-) AMD Opteron doesn't respect "pause" instruction at all (as it is written in s_lock.h , and several times mentioned in Internet). Given whole reason of `perfrom_spin_delay` is to spent time somewhere, making it a tiny-bit slower may only improve things :-) On 2017-10-19 02:28, Andres Freund wrote: > On 2017-06-05 16:22:58 +0300, Sokolov Yura wrote: >> Algorithm for LWLockWaitForVar is also refactored. New version is: >> 1. If lock is not held by anyone, it immediately exit. >> 2. Otherwise it is checked for ability to take WaitList lock, because >> variable is protected with it. If so, CAS is performed, and if it is >> successful, loop breaked to step 4. >> 3. Otherwise spin_delay perfomed, and loop returns to step 1. >> 4. var is checked for change. >> If it were changed, we unlock wait list lock and exit. >> Note: it could not change in following steps because we are holding >> wait list lock. >> 5. Otherwise CAS on setting necessary flags is performed. If it >> succeed, >> then queue Proc to wait list and exit. >> 6. If CAS failed, then there is possibility for LWLock to be already >> released - if so then we should unlock wait list and exit. >> 7. Otherwise loop returns to step 5. >> >> So invariant is preserved: >> - either we found LWLock free, >> - or we found changed variable, >> - or we set flags and queue self while LWLock were held. >> >> Spin_delay is not performed at step 7, because we should release wait >> list lock as soon as possible. > > That seems unconvincing - by not delaying you're more likely to > *increase* the time till the current locker that holds the lock can > release the lock. But why? If our CAS wasn't satisfied, then other's one were satisfied. So there is always overall progress. And if it will sleep in this loop, then other waiters will spin in first loop in this functions. But given concrete usage of LWLockWaitForVar, probably it is not too bad to hold other waiters in first loop in this function (they loops until we release WaitListLock or lock released at whole). I'm in doubts what is better. May I keep it unchanged now? BTW, I found a small mistake in this place: I forgot to set LW_FLAG_LOCKED in a state before this CAS. Looks like it wasn't real error, because CAS always failed at first loop iteration (because real `lock->state` had LW_FLAG_LOCKED already set), and after failed CAS state adsorbs value from `lock->state`. I'll fix it. > One big advantage of this is that we should be able to make LW_SHARED > acquisition an xadd. I'd done that previously, when the separate > spinlock still existed, and it was quite beneficial for performance on > larger systems. But it was some fiddly code - should be easier now. > Requires some careful management when noticing that an xadd acquired a > shared lock even though there's a conflicting exclusive lock, but > increase the fastpath. -- Sokolov Yura aka funny_falcon Postgres Professional: https://postgrespro.ru The Russian Postgres Company -- Sent via pgsql-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers ^ permalink raw reply [nested|flat] 62+ messages in thread
* Re: Fix performance degradation of contended LWLock on NUMA @ 2017-10-19 12:10 Sokolov Yura <[email protected]> parent: Sokolov Yura <[email protected]> 1 sibling, 0 replies; 62+ messages in thread From: Sokolov Yura @ 2017-10-19 12:10 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: [email protected]; pgsql-hackers > On 2017-10-19 02:28, Andres Freund wrote: >> On 2017-06-05 16:22:58 +0300, Sokolov Yura wrote: >>> Algorithm for LWLockWaitForVar is also refactored. New version is: >>> 1. If lock is not held by anyone, it immediately exit. >>> 2. Otherwise it is checked for ability to take WaitList lock, because >>> variable is protected with it. If so, CAS is performed, and if it >>> is >>> successful, loop breaked to step 4. >>> 3. Otherwise spin_delay perfomed, and loop returns to step 1. >>> 4. var is checked for change. >>> If it were changed, we unlock wait list lock and exit. >>> Note: it could not change in following steps because we are holding >>> wait list lock. >>> 5. Otherwise CAS on setting necessary flags is performed. If it >>> succeed, >>> then queue Proc to wait list and exit. >>> 6. If CAS failed, then there is possibility for LWLock to be already >>> released - if so then we should unlock wait list and exit. >>> 7. Otherwise loop returns to step 5. >>> >>> So invariant is preserved: >>> - either we found LWLock free, >>> - or we found changed variable, >>> - or we set flags and queue self while LWLock were held. >>> >>> Spin_delay is not performed at step 7, because we should release wait >>> list lock as soon as possible. >> >> That seems unconvincing - by not delaying you're more likely to >> *increase* the time till the current locker that holds the lock can >> release the lock. > > But why? If our CAS wasn't satisfied, then other's one were satisfied. > So there is always overall progress. And if it will sleep in this > loop, then other waiters will spin in first loop in this functions. > But given concrete usage of LWLockWaitForVar, probably it is not too > bad to hold other waiters in first loop in this function (they loops > until we release WaitListLock or lock released at whole). > > I'm in doubts what is better. May I keep it unchanged now? In fact, if waiter will sleep here, lock holder will not be able to set variable we are waiting for, and therefore will not release lock. It is stated in the comment for the loop: + * Note: value could not change again cause we are holding WaitList lock. So delaying here we certainly will degrade. > > BTW, I found a small mistake in this place: I forgot to set > LW_FLAG_LOCKED in a state before this CAS. Looks like it wasn't real > error, because CAS always failed at first loop iteration (because real > `lock->state` had LW_FLAG_LOCKED already set), and after failed CAS > state adsorbs value from `lock->state`. > I'll fix it. Attach contains version with a fix. -- Sokolov Yura aka funny_falcon Postgres Professional: https://postgrespro.ru The Russian Postgres Company -- Sent via pgsql-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers Attachments: [application/x-gzip] lwlock_v4.patch.gz (9.6K, ../../[email protected]/2-lwlock_v4.patch.gz) download ^ permalink raw reply [nested|flat] 62+ messages in thread
* Re: Fix performance degradation of contended LWLock on NUMA @ 2017-10-19 16:46 Andres Freund <[email protected]> parent: Sokolov Yura <[email protected]> 1 sibling, 1 reply; 62+ messages in thread From: Andres Freund @ 2017-10-19 16:46 UTC (permalink / raw) To: Sokolov Yura <[email protected]>; +Cc: [email protected]; pgsql-hackers On 2017-10-19 14:36:56 +0300, Sokolov Yura wrote: > > > + init_local_spin_delay(&delayStatus); > > > > The way you moved this around has the disadvantage that we now do this - > > a number of writes - even in the very common case where the lwlock can > > be acquired directly. > > Excuse me, I don't understand fine. > Do you complain against init_local_spin_delay placed here? Yes. > Placing it in other place will complicate code. > Or you complain against setting `mask` and `add`? That seems right. > In both cases, I think simpler version should be accepted first. It acts > as algorithm definition. And it already gives measurable improvement. Well, in scalability. I'm less sure about uncontended performance. > > > + * We intentionally do not call finish_spin_delay here, because > > > the loop > > > + * above usually finished by queuing into the wait list on > > > contention, and > > > + * doesn't reach spins_per_delay thereby doesn't sleep inside of > > > + * perform_spin_delay. Also, different LWLocks has very different > > > + * contention pattern, and it is wrong to update spin-lock > > > statistic based > > > + * on LWLock contention. > > > + */ > > > > Huh? This seems entirely unconvincing. Without adjusting this here we'll > > just spin the same way every iteration. Especially for the case where > > somebody else holds LW_FLAG_LOCKED that's quite bad. > > LWLock's are very different. Some of them are always short-term > (BufferLock), others are always locked for a long time. That seems not particularly relevant. The same is true for spinlocks. The relevant question isn't how long the lwlock is held, it's how long LW_FLAG_LOCKED is held - and that should only depend on contention (i.e. bus speed, amount of times put into sleep while holding lock, etc), not on how long the lock is held. > I've tried to place this delay into lock itself (it has 2 free bytes), > but this attempt performed worse. That seems unsurprising - there's a *lot* of locks, and we'd have to tune all of them. Additionally there's a bunch of platforms where we do *not* have free bytes (consider the embedding in BufferTag). > Now I understand, that delays should be stored in array indexed by > tranche. But I have no time to test this idea. And I doubt it will give > cardinally better results (ie > 5%), so I think it is better to accept > patch in this way, and then experiment with per-tranche delay. I don't think tranches have any decent predictive power here. Greetings, Andres Freund -- Sent via pgsql-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers ^ permalink raw reply [nested|flat] 62+ messages in thread
* Re: Fix performance degradation of contended LWLock on NUMA @ 2017-10-20 08:54 Sokolov Yura <[email protected]> parent: Andres Freund <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Sokolov Yura @ 2017-10-20 08:54 UTC (permalink / raw) To: Andres Freund <[email protected]>; +Cc: [email protected]; pgsql-hackers; [email protected] Hello, On 2017-10-19 19:46, Andres Freund wrote: > On 2017-10-19 14:36:56 +0300, Sokolov Yura wrote: >> > > + init_local_spin_delay(&delayStatus); >> > >> > The way you moved this around has the disadvantage that we now do this - >> > a number of writes - even in the very common case where the lwlock can >> > be acquired directly. >> >> Excuse me, I don't understand fine. >> Do you complain against init_local_spin_delay placed here? > > Yes. I could place it before perform_spin_delay under `if (!spin_inited)` if you think it is absolutely must. > > >> Placing it in other place will complicate code. > > >> Or you complain against setting `mask` and `add`? > > That seems right. > > >> In both cases, I think simpler version should be accepted first. It >> acts >> as algorithm definition. And it already gives measurable improvement. > > Well, in scalability. I'm less sure about uncontended performance. > > > >> > > + * We intentionally do not call finish_spin_delay here, because >> > > the loop >> > > + * above usually finished by queuing into the wait list on >> > > contention, and >> > > + * doesn't reach spins_per_delay thereby doesn't sleep inside of >> > > + * perform_spin_delay. Also, different LWLocks has very different >> > > + * contention pattern, and it is wrong to update spin-lock >> > > statistic based >> > > + * on LWLock contention. >> > > + */ >> > >> > Huh? This seems entirely unconvincing. Without adjusting this here we'll >> > just spin the same way every iteration. Especially for the case where >> > somebody else holds LW_FLAG_LOCKED that's quite bad. >> >> LWLock's are very different. Some of them are always short-term >> (BufferLock), others are always locked for a long time. > > That seems not particularly relevant. The same is true for > spinlocks. The relevant question isn't how long the lwlock is held, > it's > how long LW_FLAG_LOCKED is held - and that should only depend on > contention (i.e. bus speed, amount of times put into sleep while > holding > lock, etc), not on how long the lock is held. > >> I've tried to place this delay into lock itself (it has 2 free bytes), >> but this attempt performed worse. > > That seems unsurprising - there's a *lot* of locks, and we'd have to > tune all of them. Additionally there's a bunch of platforms where we do > *not* have free bytes (consider the embedding in BufferTag). > > >> Now I understand, that delays should be stored in array indexed by >> tranche. But I have no time to test this idea. And I doubt it will >> give >> cardinally better results (ie > 5%), so I think it is better to accept >> patch in this way, and then experiment with per-tranche delay. > > I don't think tranches have any decent predictive power here. Look after "Make acquiring LWLock to look more like spinlock". First `skip_wait_list` iterations there is no attempt to queue self into wait list. It gives most of improvement. (without it there is just no degradation on high number of clients, but a little decline on low clients, because current algorithm is also a little `spinny` (because it attempts to acquire lock again after queueing into wait list)). `skip_wait_list` depends on tranche very much. And per-tranche `skip_wait_list` should be calculated based on ability to acquire lock without any sleep, ie without both queuing itself into wait list and falling into `pg_usleep` inside `perform_spin_delay` . I suppose, it is obvious that `spins_per_delay` have to be proportional to `skip_wait_list` (it should be at least greater than `skip_wait_list`). Therefore `spins_per_delay` is also should be runtime-dependent on lock's tranche. Am I wrong? Without that per-tranche auto-tuning, it is better to not touch `spins_per_delay` inside of `LWLockAttemptLockOrQueue`, I think. With regards, -- Sokolov Yura aka funny_falcon Postgres Professional: https://postgrespro.ru The Russian Postgres Company -- Sent via pgsql-hackers mailing list ([email protected]) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH 1/3] anyarray_anyelement_operators @ 2021-03-13 10:01 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-13 10:01 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 43 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 375 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index b7150510ab..9a3f79e3b7 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17525,6 +17525,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..b10bd04ec8 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,33 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = &PG_GETARG_DATUM(0); + nulls = &PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +137,14 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + /* + * only items that match the queried element + * are considered candidate + */ + + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -185,6 +204,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* we will need recheck */ *recheck = true; @@ -274,6 +294,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } } break; + case GinContainsElemStrategy: case GinContainedStrategy: /* can't do anything else useful here */ res = GIN_MAYBE; diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index f7012cc5d9..f8cbf64c9e 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 0d4eac8f96..7ef071135c 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 4e0c9be58c..8bc05707c7 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8180,6 +8180,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 8bc7721e7d..95c9ae5443 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..698d322e14 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 3 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 0 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index c40619a8d5..b5eec945f7 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.17.0 --opg8F0UgoHELSI+9 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="0002-fix.patch" ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* [PATCH v10 1/2] anyarray_anyelement_operators @ 2021-03-15 15:10 Mark Rofail <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Mark Rofail @ 2021-03-15 15:10 UTC (permalink / raw) --- doc/src/sgml/func.sgml | 28 +++++ doc/src/sgml/gin.sgml | 8 +- doc/src/sgml/indices.sgml | 2 +- src/backend/access/gin/ginarrayproc.c | 40 +++++-- src/backend/utils/adt/arrayfuncs.c | 137 +++++++++++++++++++++++ src/include/catalog/pg_amop.dat | 3 + src/include/catalog/pg_operator.dat | 14 ++- src/include/catalog/pg_proc.dat | 6 + src/test/regress/expected/arrays.out | 92 +++++++++++++++ src/test/regress/expected/gin.out | 34 ++++++ src/test/regress/expected/opr_sanity.out | 6 +- src/test/regress/sql/arrays.sql | 10 ++ src/test/regress/sql/gin.sql | 8 ++ 13 files changed, 372 insertions(+), 16 deletions(-) diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 9492a3c6b9..04216e96a3 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -17539,6 +17539,34 @@ SELECT NULLIF(value, '(none)') ... </para></entry> </row> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyarray</type> <literal>@>></literal> <type>anyelement</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Does the array contain the specified element? + </para> + <para> + <literal>ARRAY[1,4,3] @>> 3</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <type>anyelement</type> <literal><<@</literal> <type>anyarray</type> + <returnvalue>boolean</returnvalue> + </para> + <para> + Is the specified element contained in the array? + </para> + <para> + <literal>2 <<@ ARRAY[1,7,4,2,6]</literal> + <returnvalue>t</returnvalue> + </para></entry> + </row> + <row> <entry role="func_table_entry"><para role="func_signature"> <type>anyarray</type> <literal>&&</literal> <type>anyarray</type> diff --git a/doc/src/sgml/gin.sgml b/doc/src/sgml/gin.sgml index d68d12d515..981513b765 100644 --- a/doc/src/sgml/gin.sgml +++ b/doc/src/sgml/gin.sgml @@ -84,7 +84,7 @@ </thead> <tbody> <row> - <entry morerows="3" valign="middle"><literal>array_ops</literal></entry> + <entry morerows="5" valign="middle"><literal>array_ops</literal></entry> <entry><literal>&& (anyarray,anyarray)</literal></entry> </row> <row> @@ -93,6 +93,12 @@ <row> <entry><literal><@ (anyarray,anyarray)</literal></entry> </row> + <row> + <entry><literal>@>> (anyarray,anyelement)</literal></entry> + </row> + <row> + <entry><literal><<@ (anyelement,anyarray)</literal></entry> + </row> <row> <entry><literal>= (anyarray,anyarray)</literal></entry> </row> diff --git a/doc/src/sgml/indices.sgml b/doc/src/sgml/indices.sgml index 623962d1d8..6de6c33c75 100644 --- a/doc/src/sgml/indices.sgml +++ b/doc/src/sgml/indices.sgml @@ -326,7 +326,7 @@ SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10; for arrays, which supports indexed queries using these operators: <synopsis> -<@ @> = && +<@ @> <<@ @>> = && </synopsis> (See <xref linkend="functions-array"/> for the meaning of diff --git a/src/backend/access/gin/ginarrayproc.c b/src/backend/access/gin/ginarrayproc.c index bf73e32932..eb7d13dbba 100644 --- a/src/backend/access/gin/ginarrayproc.c +++ b/src/backend/access/gin/ginarrayproc.c @@ -24,6 +24,7 @@ #define GinContainsStrategy 2 #define GinContainedStrategy 3 #define GinEqualStrategy 4 +#define GinContainsElemStrategy 5 /* @@ -78,8 +79,6 @@ ginarrayextract_2args(PG_FUNCTION_ARGS) Datum ginqueryarrayextract(PG_FUNCTION_ARGS) { - /* Make copy of array input to ensure it doesn't disappear while in use */ - ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); int32 *nkeys = (int32 *) PG_GETARG_POINTER(1); StrategyNumber strategy = PG_GETARG_UINT16(2); @@ -87,21 +86,35 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) /* Pointer *extra_data = (Pointer *) PG_GETARG_POINTER(4); */ bool **nullFlags = (bool **) PG_GETARG_POINTER(5); int32 *searchMode = (int32 *) PG_GETARG_POINTER(6); - int16 elmlen; - bool elmbyval; - char elmalign; Datum *elems; bool *nulls; int nelems; - get_typlenbyvalalign(ARR_ELEMTYPE(array), - &elmlen, &elmbyval, &elmalign); + if (strategy == GinContainsElemStrategy) + { + /* single element is passed, set elems to its pointer */ + elems = palloc(sizeof(*elems)); + *elems = PG_GETARG_DATUM(0); + nulls = palloc(sizeof(*nulls)); + *nulls = PG_ARGISNULL(0); + nelems = 1; + } + else + { + /* Make copy of array input to ensure it doesn't disappear while in use */ + ArrayType *array = PG_GETARG_ARRAYTYPE_P_COPY(0); + int16 elmlen; + bool elmbyval; + char elmalign; - deconstruct_array(array, - ARR_ELEMTYPE(array), - elmlen, elmbyval, elmalign, - &elems, &nulls, &nelems); + get_typlenbyvalalign(ARR_ELEMTYPE(array), + &elmlen, &elmbyval, &elmalign); + deconstruct_array(array, + ARR_ELEMTYPE(array), + elmlen, elmbyval, elmalign, + &elems, &nulls, &nelems); + } *nkeys = nelems; *nullFlags = nulls; @@ -126,6 +139,9 @@ ginqueryarrayextract(PG_FUNCTION_ARGS) else *searchMode = GIN_SEARCH_MODE_INCLUDE_EMPTY; break; + case GinContainsElemStrategy: + *searchMode = GIN_SEARCH_MODE_DEFAULT; + break; default: elog(ERROR, "ginqueryarrayextract: unknown strategy number: %d", strategy); @@ -172,6 +188,7 @@ ginarrayconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* result is not lossy */ *recheck = false; /* must have all elements in check[] true, and no nulls */ @@ -259,6 +276,7 @@ ginarraytriconsistent(PG_FUNCTION_ARGS) } break; case GinContainsStrategy: + case GinContainsElemStrategy: /* must have all elements in check[] true, and no nulls */ res = GIN_TRUE; for (i = 0; i < nkeys; i++) diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c index 17a16b4c5c..518d3aaaf9 100644 --- a/src/backend/utils/adt/arrayfuncs.c +++ b/src/backend/utils/adt/arrayfuncs.c @@ -4328,6 +4328,143 @@ arraycontained(PG_FUNCTION_ARGS) } +/* + * array_contains_elem : checks an array for a specific element + * adapted from array_contain_compare() for containment of a single element + */ +static bool +array_contains_elem(AnyArrayType *array, Datum elem, Oid elemtype, + Oid collation, void **fn_extra) +{ + LOCAL_FCINFO(locfcinfo, 2); + Oid arrtype = AARR_ELEMTYPE(array); + TypeCacheEntry *typentry; + int nelems; + int typlen; + bool typbyval; + char typalign; + int i; + array_iter it; + + if (arrtype != elemtype) + ereport(ERROR, + (errcode(ERRCODE_DATATYPE_MISMATCH), + errmsg("cannot compare arrays elements with element of different type"))); + + /* + * We arrange to look up the equality function only once per series of + * calls, assuming the element type doesn't change underneath us. The + * typcache is used so that we have no memory leakage when being used as + * an index support function. + */ + typentry = (TypeCacheEntry *) *fn_extra; + if (typentry == NULL || + typentry->type_id != arrtype) + { + typentry = lookup_type_cache(arrtype, + TYPECACHE_EQ_OPR_FINFO); + if (!OidIsValid(typentry->eq_opr_finfo.fn_oid)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_FUNCTION), + errmsg("could not identify an equality operator for type %s", + format_type_be(arrtype)))); + *fn_extra = (void *) typentry; + } + typlen = typentry->typlen; + typbyval = typentry->typbyval; + typalign = typentry->typalign; + + /* + * Apply the comparison operator for the passed element against each + * element in the array + */ + InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2, + collation, NULL, NULL); + + /* Loop over source data */ + nelems = ArrayGetNItems(AARR_NDIM(array), AARR_DIMS(array)); + array_iter_setup(&it, array); + + for (i = 0; i < nelems; i++) + { + Datum elt; + bool isnull; + bool oprresult; + + /* Get element, checking for NULL */ + elt = array_iter_next(&it, &isnull, i, typlen, typbyval, typalign); + + /* + * We assume that the comparison operator is strict, so a NULL can't + * match anything. refer to the comment in array_contain_compare() + */ + if (isnull) + continue; + + /* + * Apply the operator to the element pair; treat NULL as false + */ + locfcinfo->args[0].value = elt; + locfcinfo->args[0].isnull = false; + locfcinfo->args[1].value = elem; + locfcinfo->args[1].isnull = false; + locfcinfo->isnull = false; + oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo)); + if (!locfcinfo->isnull && oprresult) + return true; + } + + return false; +} + +Datum +arraycontainselem(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0); + Datum elem = PG_GETARG_DATUM(1); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 1); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 0); + + PG_RETURN_BOOL(result); +} + +Datum +arrayelemcontained(PG_FUNCTION_ARGS) +{ + AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(1); + Datum elem = PG_GETARG_DATUM(0); + Oid elemtype = get_fn_expr_argtype(fcinfo->flinfo, 0); + Oid collation = PG_GET_COLLATION(); + bool result; + + /* + * we don't need to check if the elem is null or if the elem datatype and + * array datatype match since this is handled within internal calls already + * (a property of polymorphic functions) + */ + + result = array_contains_elem(array, elem, elemtype, collation, + &fcinfo->flinfo->fn_extra); + + /* Avoid leaking memory when handed toasted input */ + AARR_FREE_IF_COPY(array, 1); + + PG_RETURN_BOOL(result); +} + /*----------------------------------------------------------------------------- * Array iteration functions * These functions are used to iterate efficiently through arrays diff --git a/src/include/catalog/pg_amop.dat b/src/include/catalog/pg_amop.dat index 0f7ff63669..8a14fc7140 100644 --- a/src/include/catalog/pg_amop.dat +++ b/src/include/catalog/pg_amop.dat @@ -1242,6 +1242,9 @@ { amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', amoprighttype => 'anyarray', amopstrategy => '4', amopopr => '=(anyarray,anyarray)', amopmethod => 'gin' }, +{ amopfamily => 'gin/array_ops', amoplefttype => 'anyarray', + amoprighttype => 'anyelement', amopstrategy => '5', + amopopr => '@>>(anyarray,anyelement)', amopmethod => 'gin' }, # btree enum_ops { amopfamily => 'btree/enum_ops', amoplefttype => 'anyenum', diff --git a/src/include/catalog/pg_operator.dat b/src/include/catalog/pg_operator.dat index 85395a81ee..c6e809b88d 100644 --- a/src/include/catalog/pg_operator.dat +++ b/src/include/catalog/pg_operator.dat @@ -2761,7 +2761,7 @@ oprresult => 'bool', oprcode => 'circle_overabove', oprrest => 'positionsel', oprjoin => 'positionjoinsel' }, -# overlap/contains/contained for arrays +# overlap/contains/contained/elemcontained/containselem for arrays { oid => '2750', oid_symbol => 'OID_ARRAY_OVERLAP_OP', descr => 'overlaps', oprname => '&&', oprleft => 'anyarray', oprright => 'anyarray', oprresult => 'bool', oprcom => '&&(anyarray,anyarray)', @@ -2778,6 +2778,18 @@ oprresult => 'bool', oprcom => '@>(anyarray,anyarray)', oprcode => 'arraycontained', oprrest => 'arraycontsel', oprjoin => 'arraycontjoinsel' }, +{ oid => '6108', oid_symbol => 'OID_ARRAY_ELEMCONTAINED_OP', + descr => 'elem is contained by', + oprname => '<<@', oprleft => 'anyelement', oprright => 'anyarray', + oprresult => 'bool', oprcom => '@>>(anyarray,anyelement)', + oprcode => 'arrayelemcontained', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, +{ oid => '6105', oid_symbol => 'OID_ARRAY_CONTAINSELEM_OP', + descr => 'contains elem', + oprname => '@>>', oprleft => 'anyarray', oprright => 'anyelement', + oprresult => 'bool', oprcom => '<<@(anyelement,anyarray)', + oprcode => 'arraycontainselem', oprrest => 'arraycontsel', + oprjoin => 'arraycontjoinsel' }, # capturing operators to preserve pre-8.3 behavior of text concatenation { oid => '2779', descr => 'concatenate', diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 93393fcfd4..8d82e64f86 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -8200,6 +8200,12 @@ { oid => '2749', proname => 'arraycontained', prorettype => 'bool', proargtypes => 'anyarray anyarray', prosrc => 'arraycontained' }, +{ oid => '6109', + proname => 'arrayelemcontained', prorettype => 'bool', + proargtypes => 'anyelement anyarray', prosrc => 'arrayelemcontained' }, +{ oid => '6107', + proname => 'arraycontainselem', prorettype => 'bool', + proargtypes => 'anyarray anyelement', prosrc => 'arraycontainselem' }, # BRIN minmax { oid => '3383', descr => 'BRIN minmax support', diff --git a/src/test/regress/expected/arrays.out b/src/test/regress/expected/arrays.out index 3e3a1beaab..03ce07e219 100644 --- a/src/test/regress/expected/arrays.out +++ b/src/test/regress/expected/arrays.out @@ -758,6 +758,28 @@ SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} (6 rows) +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 74 | {32} | {AAAAAAAAAAAAAAAA1729,AAAAAAAAAAAAA22860,AAAAAA99807,AAAAA17383,AAAAAAAAAAAAAAA67062,AAAAAAAAAAA15165,AAAAAAAAAAA50956} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} + 98 | {38,34,32,89} | {AAAAAAAAAAAAAAAAAA71621,AAAA8857,AAAAAAAAAAAAAAAAAAA65037,AAAAAAAAAAAAAAAA31334,AAAAAAAAAA48845} + 100 | {85,32,57,39,49,84,32,3,30} | {AAAAAAA80240,AAAAAAAAAAAAAAAA1729,AAAAA60038,AAAAAAAAAAA92631,AAAAAAAA9523} +(6 rows) + SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -782,6 +804,32 @@ SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} (8 rows) +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; + seqno | i | t +-------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ + 6 | {39,35,5,94,17,92,60,32} | {AAAAAAAAAAAAAAA35875,AAAAAAAAAAAAAAAA23657} + 12 | {17,99,18,52,91,72,0,43,96,23} | {AAAAA33250,AAAAAAAAAAAAAAAAAAA85420,AAAAAAAAAAA33576} + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 19 | {52,82,17,74,23,46,69,51,75} | {AAAAAAAAAAAAA73084,AAAAA75968,AAAAAAAAAAAAAAAA14047,AAAAAAA80240,AAAAAAAAAAAAAAAAAAA1205,A68938} + 53 | {38,17} | {AAAAAAAAAAA21658} + 65 | {61,5,76,59,17} | {AAAAAA99807,AAAAA64741,AAAAAAAAAAA53908,AA21643,AAAAAAAAA10012} + 77 | {97,15,32,17,55,59,18,37,50,39} | {AAAAAAAAAAAA67946,AAAAAA54032,AAAAAAAA81587,55847,AAAAAAAAAAAAAA28620,AAAAAAAAAAAAAAAAA43052,AAAAAA75463,AAAA49534,AAAAAAAA44066} + 89 | {40,32,17,6,30,88} | {AA44673,AAAAAAAAAAA6119,AAAAAAAAAAAAAAAA23657,AAAAAAAAAAAAAAAAAA47955,AAAAAAAAAAAAAAAA33598,AAAAAAAAAAA33576,AA44673} +(8 rows) + SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; seqno | i | t -------+---------------------------------+------------------------------------------------------------------------------------------------------------------------------------ @@ -963,6 +1011,16 @@ SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; -------+---+--- (0 rows) +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; + seqno | i | t +-------+---+--- +(0 rows) + SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; seqno | i | t -------+---+--- @@ -983,6 +1041,24 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} (4 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; + seqno | i | t +-------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- + 22 | {11,6,56,62,53,30} | {AAAAAAAA72908} + 45 | {99,45} | {AAAAAAAA72908,AAAAAAAAAAAAAAAAAAA17075,AA88409,AAAAAAAAAAAAAAAAAA36842,AAAAAAA48038,AAAAAAAAAAAAAA10611} + 72 | {22,1,16,78,20,91,83} | {47735,AAAAAAA56483,AAAAAAAAAAAAA93788,AA42406,AAAAAAAAAAAAA73084,AAAAAAAA72908,AAAAAAAAAAAAAAAAAA61286,AAAAA66674,AAAAAAAAAAAAAAAAA50407} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} +(4 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; seqno | i | t -------+-----------------------+-------------------------------------------------------------------------------------------------------------------------------------------- @@ -1000,6 +1076,22 @@ SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; 96 | {23,97,43} | {AAAAAAAAAA646,A87088} (3 rows) +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; + seqno | i | t +-------+------------------+-------------------------------------------------------------------- + 15 | {17,14,16,63,67} | {AA6416,AAAAAAAAAA646,AAAAA95309} + 79 | {45} | {AAAAAAAAAA646,AAAAAAAAAAAAAAAAAAA70415,AAAAAA43678,AAAAAAAA72908} + 96 | {23,97,43} | {AAAAAAAAAA646,A87088} +(3 rows) + SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; seqno | i | t -------+------------------+-------------------------------------------------------------------- diff --git a/src/test/regress/expected/gin.out b/src/test/regress/expected/gin.out index 6402e89c7f..7fc7436646 100644 --- a/src/test/regress/expected/gin.out +++ b/src/test/regress/expected/gin.out @@ -53,6 +53,40 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; 3 (1 row) +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 1) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 1) +(5 rows) + +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + QUERY PLAN +----------------------------------------------- + Aggregate + -> Bitmap Heap Scan on gin_test_tbl + Recheck Cond: (i @>> 999) + -> Bitmap Index Scan on gin_test_idx + Index Cond: (i @>> 999) +(5 rows) + +select count(*) from gin_test_tbl where i @>> 1; + count +------- + 2997 +(1 row) + +select count(*) from gin_test_tbl where i @>> 999; + count +------- + 3 +(1 row) + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; explain (costs off) diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 254ca06d3d..5de5ab6d13 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -1173,6 +1173,7 @@ ORDER BY 1, 2; <-> | <-> << | >> <<= | >>= + <<@ | @>> <= | >= <> | <> <@ | @> @@ -1188,7 +1189,7 @@ ORDER BY 1, 2; ~<=~ | ~>=~ ~<~ | ~>~ ~= | ~= -(29 rows) +(30 rows) -- Likewise for negator pairs. SELECT DISTINCT o1.oprname AS op1, o2.oprname AS op2 @@ -2029,6 +2030,7 @@ ORDER BY 1, 2, 3; 2742 | 2 | @@@ 2742 | 3 | <@ 2742 | 4 | = + 2742 | 5 | @>> 2742 | 7 | @> 2742 | 9 | ? 2742 | 10 | ?| @@ -2100,7 +2102,7 @@ ORDER BY 1, 2, 3; 4000 | 28 | ^@ 4000 | 29 | <^ 4000 | 30 | >^ -(123 rows) +(124 rows) -- Check that all opclass search operators have selectivity estimators. -- This is not absolutely required, but it seems a reasonable thing diff --git a/src/test/regress/sql/arrays.sql b/src/test/regress/sql/arrays.sql index 912233ef96..944fa3afdd 100644 --- a/src/test/regress/sql/arrays.sql +++ b/src/test/regress/sql/arrays.sql @@ -319,8 +319,12 @@ SELECT 0 || ARRAY[1,2] || 3 AS "{0,1,2,3}"; SELECT ARRAY[1.1] || ARRAY[2,3,4]; SELECT * FROM array_op_test WHERE i @> '{32}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 32 ORDER BY seqno; +SELECT * FROM array_op_test WHERE 32 <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{17}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 17 <<@ i ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> 17 ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{32,17}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{32,17}' ORDER BY seqno; @@ -331,12 +335,18 @@ SELECT * FROM array_op_test WHERE i && '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i = '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i @> '{NULL}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE i @>> NULL ORDER BY seqno; +SELECT * FROM array_op_test WHERE NULL <<@ i ORDER BY seqno; SELECT * FROM array_op_test WHERE i && '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE i <@ '{NULL}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAA72908' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAA72908' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAAAA646}' ORDER BY seqno; +SELECT * FROM array_op_test WHERE t @>> 'AAAAAAAAAA646' ORDER BY seqno; +SELECT * FROM array_op_test WHERE 'AAAAAAAAAA646' <<@ t ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t @> '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; SELECT * FROM array_op_test WHERE t && '{AAAAAAAA72908,AAAAAAAAAA646}' ORDER BY seqno; diff --git a/src/test/regress/sql/gin.sql b/src/test/regress/sql/gin.sql index 5194afcc1f..c9b40903c6 100644 --- a/src/test/regress/sql/gin.sql +++ b/src/test/regress/sql/gin.sql @@ -41,6 +41,14 @@ select count(*) from gin_test_tbl where i @> array[1, 999]; select count(*) from gin_test_tbl where i @> array[1, 999]; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 1; +explain (costs off) +select count(*) from gin_test_tbl where i @>> 999; + +select count(*) from gin_test_tbl where i @>> 1; +select count(*) from gin_test_tbl where i @>> 999; + -- Very weak test for gin_fuzzy_search_limit set gin_fuzzy_search_limit = 1000; -- 2.30.1 --------------3A927D779D0A0B91F132AD46-- ^ permalink raw reply [nested|flat] 62+ messages in thread
* WaitForOlderSnapshots in DETACH PARTITION causes deadlocks @ 2023-07-24 19:40 Imseih (AWS), Sami <[email protected]> 0 siblings, 1 reply; 62+ messages in thread From: Imseih (AWS), Sami @ 2023-07-24 19:40 UTC (permalink / raw) To: PostgreSQL Hackers <[email protected]> Hi, While recently looking into partition maintenance, I found a case in which DETACH PARTITION FINALIZE could case deadlocks. This occurs when a ALTER TABLE DETACH CONCURRENTLY, followed by a cancel, the followed by an ALTER TABLE DETACH FINALIZE, and this sequence of steps occur in the middle of other REPEATABLE READ/SERIALIZABLE transactions. These RR/SERIALIZABLE should not accessed the partition until the FINALIZE step starts. See the attached repro. This seems to occur as the FINALIZE is calling WaitForOlderSnapshot [1] to make sure that all older snapshots are completed before the finalize is completed and the detached partition is removed from the parent table association. WaitForOlderSnapshots is used here to ensure that snapshots older than the start of the ALTER TABLE DETACH CONCURRENTLY are completely removed to guarantee consistency, however it does seem to cause deadlocks for at least RR/SERIALIZABLE transactions. There are cases [2] in which certain operations are accepted as not being MVCC-safe, and now I am wondering if this is another case? Deadlocks are not a good scenario, and WaitForOlderSnapshot does not appear to do anything for READ COMMITTED transactions. So do we actually need WaitForOlderSnapshot in the FINALIZE code? and Could be acceptable that this operation is marked as not MVCC-safe like the other aforementioned operations? Perhaps I am missing some important point here, so any feedback will be appreciated. Regards, Sami Imseih Amazon Web Services (AWS) [1] https://github.com/postgres/postgres/blob/master/src/backend/commands/tablecmds.c#L18757-L18764 [2] https://www.postgresql.org/docs/current/mvcc-caveats.html Attachments: [application/octet-stream] partition_detach_finalize_deadlock_repro.sql (1.9K, ../../[email protected]/3-partition_detach_finalize_deadlock_repro.sql) download ^ permalink raw reply [nested|flat] 62+ messages in thread
* Re: WaitForOlderSnapshots in DETACH PARTITION causes deadlocks @ 2023-07-25 07:03 Michael Paquier <[email protected]> parent: Imseih (AWS), Sami <[email protected]> 0 siblings, 1 reply; 62+ messages in thread From: Michael Paquier @ 2023-07-25 07:03 UTC (permalink / raw) To: Imseih (AWS), Sami <[email protected]>; +Cc: PostgreSQL Hackers <[email protected]>; Alvaro Herrera <[email protected]> On Mon, Jul 24, 2023 at 07:40:04PM +0000, Imseih (AWS), Sami wrote: > WaitForOlderSnapshots is used here to ensure that snapshots older than > the start of the ALTER TABLE DETACH CONCURRENTLY are completely removed > to guarantee consistency, however it does seem to cause deadlocks for at > least RR/SERIALIZABLE transactions. > > There are cases [2] in which certain operations are accepted as not being > MVCC-safe, and now I am wondering if this is another case? Deadlocks are > not a good scenario, and WaitForOlderSnapshot does not appear to do > anything for READ COMMITTED transactions. > > So do we actually need WaitForOlderSnapshot in the FINALIZE code? and > Could be acceptable that this operation is marked as not MVCC-safe like > the other aforementioned operations? I guess that there is an argument with lifting that a bit. Based on the test case your are providing, a deadlock occuring between the FINALIZE and a scan of the top-level partitioned table in a transaction that began before the DETACH CONCURRENTLY does not seem like the best answer to have from the user perspective. I got to wonder whether there is room to make the wait for older snapshots in the finalize phase more robust, though, and actually make it wait until the first transaction commits rather than fail because of a deadlock like that. > Perhaps I am missing some important point here, so any feedback will be > appreciated. Adding Alvaro in CC as the author of 71f4c8c6 for input, FYI. -- Michael Attachments: [application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc) download ^ permalink raw reply [nested|flat] 62+ messages in thread
* Re: WaitForOlderSnapshots in DETACH PARTITION causes deadlocks @ 2023-07-26 13:57 Alvaro Herrera <[email protected]> parent: Michael Paquier <[email protected]> 0 siblings, 0 replies; 62+ messages in thread From: Alvaro Herrera @ 2023-07-26 13:57 UTC (permalink / raw) To: Michael Paquier <[email protected]>; +Cc: Imseih (AWS), Sami <[email protected]>; PostgreSQL Hackers <[email protected]> On 2023-Jul-25, Michael Paquier wrote: > On Mon, Jul 24, 2023 at 07:40:04PM +0000, Imseih (AWS), Sami wrote: > > WaitForOlderSnapshots is used here to ensure that snapshots older than > > the start of the ALTER TABLE DETACH CONCURRENTLY are completely removed > > to guarantee consistency, however it does seem to cause deadlocks for at > > least RR/SERIALIZABLE transactions. > > > > There are cases [2] in which certain operations are accepted as not being > > MVCC-safe, and now I am wondering if this is another case? Deadlocks are > > not a good scenario, and WaitForOlderSnapshot does not appear to do > > anything for READ COMMITTED transactions. > > > > So do we actually need WaitForOlderSnapshot in the FINALIZE code? and > > Could be acceptable that this operation is marked as not MVCC-safe like > > the other aforementioned operations? > > I guess that there is an argument with lifting that a bit. Based on > the test case your are providing, a deadlock occuring between the > FINALIZE and a scan of the top-level partitioned table in a > transaction that began before the DETACH CONCURRENTLY does not seem > like the best answer to have from the user perspective. I got to > wonder whether there is room to make the wait for older snapshots in > the finalize phase more robust, though, and actually make it wait > until the first transaction commits rather than fail because of a > deadlock like that. It took a lot of work to get SERIALIZABLE/RR transactions to work with DETACHED CONCURRENTLY, so I'm certainly not in a hurry to give up and just "document that it doesn't work". I don't think that would be an acceptable feature regression. I spent some time yesterday trying to turn the reproducer into an isolationtester spec file. I have not succeeded yet, but I'll continue with that this afternoon. -- Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/ "Every machine is a smoke machine if you operate it wrong enough." https://twitter.com/libseybieda/status/1541673325781196801 ^ permalink raw reply [nested|flat] 62+ messages in thread
end of thread, other threads:[~2023-07-26 13:57 UTC | newest] Thread overview: 62+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2017-07-18 17:20 Re: Fix performance degradation of contended LWLock on NUMA Sokolov Yura <[email protected]> 2017-08-10 12:58 ` Sokolov Yura <[email protected]> 2017-09-08 15:33 ` Jesper Pedersen <[email protected]> 2017-09-08 19:35 ` Sokolov Yura <[email protected]> 2017-09-11 15:01 ` Jesper Pedersen <[email protected]> 2017-09-14 12:34 ` Jesper Pedersen <[email protected]> 2017-10-19 00:03 ` Andres Freund <[email protected]> 2017-10-19 11:36 ` Sokolov Yura <[email protected]> 2017-10-19 12:10 ` Sokolov Yura <[email protected]> 2017-10-19 16:46 ` Andres Freund <[email protected]> 2017-10-20 08:54 ` Sokolov Yura <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-13 10:01 [PATCH 1/3] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2021-03-15 15:10 [PATCH v10 1/2] anyarray_anyelement_operators Mark Rofail <[email protected]> 2023-07-24 19:40 WaitForOlderSnapshots in DETACH PARTITION causes deadlocks Imseih (AWS), Sami <[email protected]> 2023-07-25 07:03 ` Re: WaitForOlderSnapshots in DETACH PARTITION causes deadlocks Michael Paquier <[email protected]> 2023-07-26 13:57 ` Re: WaitForOlderSnapshots in DETACH PARTITION causes deadlocks Alvaro Herrera <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox