Avatar ·

Working with a large number of records in a session

📁 function, Flask microframework, AI, work, project

What is the correct way to work with a large number of records in SQLAlchemy?

I have two tables. The first one has 5 million records of the form: question_id, view_count, counted. The second table contains the sum of view_count for each unique question_id. If we have accounted for a record from the first table in the second one, counted is set to true.

Currently it looks like this:

def update_most_viewed():    query = QuestionViewHistory.query.filter_by(counted=False).distinct()    question_count = query.count()    frame_size = 1000    counter = 0