public inbox for [email protected]  

help / color / mirror / Atom feed

From: Adrian Klaver <[email protected]>
To: Robert Leach <[email protected]>
Cc: [email protected]
Subject: Re: How to perform a long running dry run transaction without blocking
Date: Fri, 7 Feb 2025 13:36:54 -0800
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
References: <[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>
	<[email protected]>

On 2/7/25 10:20, Robert Leach wrote:



> 
> Anyway, thanks so much for your help.  This discussion has been very 
> useful, and I think I will proceed at first, exactly how you suggested, 
> by queuing every validation job (using celery).  Then I will explore 
> whether or not I can apply the "on timeout" strategy in a small patch.
> 
> Incidentally, during our Wednesday meeting this week, we actually opened 
> our public instance to the world for the first time, in preparation for 
> the upcoming publication.  This discussion is about the data submission 
> interface, but that interface is actually disabled on the public-facing 
> instance.  The other part of the codebase that I was primarily 
> responsible for was the advanced search.  Everything else was primarily 
> by other team members.  If you would like to check it out, let me know 
> what you think: http://tracebase.princeton.edu 
> <http://tracebase.princeton.edu;

I would have to hit the books again to understand all of what is going 
on here. One quibble with the Download tab, there is no indication of 
the size of the datasets. I generally like to know what I am getting 
into before I start a download. Also, is there explicit throttling going 
on? I am seeing 10.2kb/sec, whereas from here 
https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page I 
downloaded a 47.65M file at 41.9MB/s

> 
> Cheers,
> Rob
> 
> 
> Robert William Leach
> Research Software Engineer
> 133 Carl C. Icahn Lab
> Lewis-Sigler Institute for Integrative Genomics
> Princeton University
> Princeton, NJ 08544
> 

-- 
Adrian Klaver
[email protected]

view thread (13+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected]
  Subject: Re: How to perform a long running dry run transaction without blocking
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox