Working with a large number of records in the session

What is the correct way to work with a large number of records with SQLAlchemy?

I have two tablets. The first 5 million records have the following types: question_id, view_count, counted. The second table contains the sum of view_count for each unique question_id. If we took into account the entry from the first table in the second, counted is set to true.

Right now it looks like this:

def update_most_viewed(): query =uery = QuestionViewHistory.query.filter_by(counted=ounted=False).distinct()    qu    question_count = query.count()    frame_size = 10001000    counter = 0    while counter <= question_count:        all_questions = query.offset(counter*frame_size).limit(frame_size)._size).all()        counter = counter + frame_size        for question in all_questions:            most_viewed_question = MostViewedQuestionuestion.query.fis="">filter_by(question_id=question.question_id).first()            if most_viewed_question is None:                m     most_viewed_question = MostViewedQuestion(question.question_id, question.view_count)                db.session.add(most_viewed_question)            else:                most_viewed_question.estion.view_count += question.view_count            question.counted = True        db.session.commit()

Calling a function from the console. Initialization:

app/span> = Flask(__name__)db="">db = SQLAlchemy(app) 

The problem is that with each pass, the time grows exponentially: after the fifth pass, everything freezes. If you run the program again, everything repeats one-on-one.

As far as I understand, the problem is that every time commit is called, SQLAlchemy updates all attributes of all objects in the session, but unfortunately I have not found a way to fix this.

Update

Model classes, which appear in the request.

class/span> MostViewedQuestionuestion(db.Model):    __tablename__ = 'most_viewed_quewed_question'    id = db.d = db.Colu"">Column(db.Integer, primary_key=True)    question_id = db.stion_id = db.Column((db.Integer)    view_counew_count = db.Column(db.Integer) span>)        is_associaassociated = db.Column(db.Boolean)    can_be_associated = db.Column((dbdb.Boolean)    title = db.Column(db.String(500))    body = db.y = db.Column>Column(db.String(30000))    tags = db.s = db.Colu"">Column(db.String(5tring(500))       l    last_update_date = date = db.Column(db.DateTime)      def __init__(self, question_id, view_count, is_associated=False):        self.n        self.question_id = question_id        self.view_count = view_count        self.is_associated = is_associated        self.can_be_associated = True        self.last_update_date = datetime.datetime.now()    def __repr__(self):        retu '<MostViewedQclass="">MostViewedQuestion %s> %s>' % str(self.idid)        class  QuestionViewHistory(db.Model):    __tablename__ = 'question_view_history'    id = db.d = db.Column>Column(db.Integer, primary_key=True)    question_id = db.stion_id = db.Column((db.Integer)    view_counew_count = db.Column(db.Integer)/span>)           view_d view_date = db.Column(db..DateTi>DateTime)     counted = db.Column(db.Boolean)    def __init__(self, question_id, view_count, view_date):        self.n        self.question_id = question_id        self.view_count = view_count        self.view_date = view_date        self.counted = False    def __repr__(self):        retu '<QuestionViewlass="">QuestionViewHistory %s> %s>' % str(self.idid)        

The code of the entire project is available on , all models are in the file, the update_most_viewed