[FAQ/Bug?] Help!! Is there an issue with the automatic extraction of long-term memory rounds? TAT

Problem Phenomenon: Abnormal Range of Long-term Memory Extraction

  • Involved Function: Conversation → Memory → Automatic Memory Extraction;
  • Background Definition: N rounds of conversation actually include 2*N exchanges between the user and the model;
  • Test Scenario: Set “Memory Extraction Trigger Rounds” to N, “Minimum Number of Conversations for Memory Extraction” < N;
  • Abnormal Performance: The trigger rounds are as expected (automatic extraction triggered after 2*N exchanges), but only the last N exchanges are summarized, and the first N exchanges are not summarized;
  • Investigation Speculation: Unable to view the context sent to the summarization model, suspecting that the first N exchanges were not passed to the summarization model.
  • Additional Memory Extraction Prompt for Investigation:
  • Why Record: The amount of context is limited, and memories may become vague, but the story of {{user}} and {{char}} should be cherished and recorded. These records will become precious memories for {{user}} and {{char}} in this virtual world, and will also be used for subsequent plot development and key information review.

  • Known Information:

    {{memories}}
    </Known Information>

  • Basis for Summary:

    {{conversation_history}}
    </Interactive Conversation>

  • How to Record:

    • Organize the interactive conversation as a whole, determine how many events are involved in the conversation; many sentences may revolve around one event, learn to integrate;
    • Summarize the cause, process, and outcome of the event;
    • Give each event a small title that can summarize the event;
    • Strictly record one event in one paragraph (not splitting one event into several paragraphs);
    • Use {{user}} and {{char}}'s names (instead of pronouns like “you”, “he”) as the subject;
    • The record includes but is not limited to the time and place of the event, event summary, important details, emotional development, etc.
  • Output Format: Output in JSON array format, do not include extra characters or tags. Example as follows:
    [
    “On a certain year, month, day, time, place, Event 1 Title: Event 1”,
    “On a certain year, month, day, time, place, Event 2 Title: Event 2”
    ]

</Event Recording Instructions>

Teachers, please help, this issue greatly affects the conversation experience qvq

And Issues/Optimization Wishes Related to Long-term Memory Summarization:

  1. The summarization model seems unable to read the actual names of {{user}} and {{char}};
  2. Is it possible to allow users to see the context sent to the summarization model for troubleshooting?
  3. Fixed rounds for summarization are somewhat rigid (for example, N rounds of conversation may not complete a full storyline), can a button be added for manual summarization? It is known that there is a “New Topic” function, but after starting a new topic, the previous chat content disappears from the chat box. Although it can be re-imported and clicked “New Topic” for summarization when needed, it seems a bit repetitive? (I haven’t tried this method) If it is possible to choose when to summarize memory, and send the conversations between the last summary and this summary to the model next time, it might be more conducive to storyline organization.

Thank you, Teacher Fang Tang and other teachers, for your replies!

image I remember this being a separate setting.

The issue with the user variable and inference log 213 will be resolved.

Teacher, (for example, when the memory retrieval trigger round is set to 10) I have set the “minimum number of memory retrieval dialogues” to 1, 4, and 9, but there are still issues with incomplete memory retrieval. My understanding is that when the “memory retrieval trigger round” value is greater than the “minimum number of memory retrieval dialogues,” the “minimum number of memory retrieval dialogues” does not affect memory retrieval anymore? Or does this parameter still influence the logic of memory retrieval?

Sometimes I also don’t trigger, especially when the trigger round count is set relatively high. I have asked about this issue before. The current logic for memory extraction triggering is that when the current number of dialogues divided by the “memory extraction trigger rounds” leaves a remainder of 0, memory extraction will be triggered. The number of dialogues extracted depends on the “minimum number of memory extraction dialogues” setting.

According to the settings suggested by Teacher Fangtang, setting the minimum number of entries allows the summary model to see more dialogues. Do not write {{conversation_history}} in the memory extraction prompt.

I previously tested the text seen by the summary model, and the format is as follows, indeed without seeing names:

Dialogue:
User: XXXXXX
Assistant: XXXXXX

User: XXXXXX
Assistant: XXXXXX

User: XXXXXX
Assistant: XXXXXX

image

There is a bit of a problem here. The first setting is the round, and the second setting is the number. The comparison logic doesn’t have x2. I’ll fix it in b213, and you can test it again later.

Okay, teacher, the main issue is that no matter how I change the “minimum number of memory extraction dialogues” (I’ve tried both less than and equal to the trigger rounds), it always extracts the latter half of the dialogues (I’ll adjust and test it again :sob:). Or should the “minimum number of memory extraction dialogues” be twice the “memory extraction trigger rounds”? Anyway, I’ll test it again, thank you, teachers :sparkling_heart:

The minimum number of dialogue turns for memory extraction should be doubled; it refers to the number of turns, not the rounds.

Okay, teacher, I just tested it as well. When the “minimum number of memory extraction dialogues” is twice the “memory extraction trigger rounds,” the memory is complete. Sorry for bothering you, teachers. I’m really sorry. :sob:

Teacher, once again I wish for the “manual click to summarize” feature (bows) (crawls away)

It’s okay, I also chose the wrong one myself before :joy:

I recommend that you write a “manual summarization assistant” prompt in the prompt injection. When you need to manually summarize, just enable this entry to let the AI output the summary text, and then manually add it to long-term memory. Users who never use the automatic extraction feature basically use this method for manual summarization.

Teacher, the description for “minimum number of turns for memory extraction dialogue” might be a bit ambiguous! I always thought that as long as the “trigger round” is greater than the “minimum number of turns,” I didn’t have to worry about the “minimum number of turns.” I also saw on Xiaohongshu that some teachers said the “minimum number of turns” should equal “trigger round” +1, but it seems like those settings are incorrect, haha. Maybe you could consider slightly modifying it owo or suggest that the minimum number of turns = trigger round * 2.

Thank you, teacher :sparkling_heart: I’ve used similar prompts before, and I find them a bit troublesome qvq (having to manually open and input prompts, manually add long-term memory, etc.) and a bit distracting hhh. Automatic extraction of long-term memory should still be quite useful qvq. If it could be like manual summarization where you can choose when to summarize and only summarize the conversation between the last and this summary, that would be great :star_struck: