Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

is there any memory leak when using arena parsing a lot of large string ? #16816

Open
lzcchl opened this issue May 10, 2024 · 1 comment
Open

Comments

@lzcchl
Copy link

lzcchl commented May 10, 2024

I have searched for a lot of answers online, but still haven't solved the problem, the answer just like:
#6197
https://stackoverflow.com/questions/50414346/how-to-free-protocol-buffer-memory/50446245#50446245

I need free the memory eagerly, so I use arena. my test code is so small, you can find at here and have a quickly reading:
to_github.zip

you can run 4 step to run my example by run "bash step1.sh" "bash step2.sh" "bash step3.sh" "bash step4.sh", if you protoc version is large than 3.21, you should change the name "CMakeLists >= v3.22.txt" to "CMakeLists.txt".

the result just like:

Hello, Protobuf!
1
free_memory_gb: 12.7 GB

2
free_memory_gb: 11.6 GB

3
free_memory_gb: 11.8 GB

4
free_memory_gb: 11.8 GB

Goodbye, Protobuf!

memory leak happen!

but!!! if you change file src/ofs/ofs.cpp and rewrite msgTest.add_str(string("1234567891234567")); to msgTest.add_str(string("123456789123456")); , the length of the string has decreased from 17 to 16, and rerun bash, the result just like:

Hello, Protobuf!
1
free_memory_gb: 12.7 GB

2
free_memory_gb: 12.1 GB

3
free_memory_gb: 12.7 GB

4
free_memory_gb: 12.7 GB

Goodbye, Protobuf!

what's a big difference! at position 2, old free_memory_gb is 11.6 GB, new free_memory_gb is 12.1 GB, and at position3 the free_memory_gb return to 12.7 GB, there are no memory leak happen!

is there any bug at arena? or should I change some parameters to avoid memory leak? I have change ArenaOptions.start_block_size and ArenaOptions.max_block_size to larger valve, but it does't work, help! please, thank you.

@zenitaanders
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants