序列化在.NET考虑streamsize限制序列化、NET、streamsize

2023-09-07 08:43:30 作者:撩荡

我序列化一个对象,其中有几个子对象(也是可序列化)。我可以检查该流的大小在序列化过程的? (我只是想限制流的大小到我序列化)

I am serializing an object, which has several sub-objects(which are also serializable). Can i check the size of the stream in between the serialization process? ( I just want to limit the size of the stream to which i am serializing )

推荐答案

根据串行器,它可能会独立地掉落物品到流的末端(与具有它仍然可以读取流回来)。不能与纯XML(由于根元素),但我知道至少有一个串行支持顺序对象读/写(不用猜它)的。

Depending on the serializer, it may be possible to drop items onto the end of the stream independently (and have it still possible to read the stream back again). Not with "pure" xml (due to the root element), but I know of at least one serializer that supports sequential object read/write (no prizes for guessing which).

如果大小的绝对的键,然后通过也许去一个的MemoryStream - 类似的信息(伪code)

If size is absolutely key, then perhaps go via a MemoryStream - something like (pseudo code)

using(Stream dest = ... )
using(MemoryStream buffer = new MemoryStream())
{
    foreach(SomeType obj in items) {
        buffer.SetLength(0); // reset to empty, but retaining buffer to reduce allocs
        someSerializer.Serialize(buffer, obj);

        if(enoughSpace) {
            // add our item
            dest.Write(buffer.GetBuffer(), 0, buffer.Length);
            // TODO: decrement space     
        } else {
            break; // or new file, whatever
        }
    }
}

完成任务。这样就避免了通过完成写作部分对象的每个的个人的对象来存储流(重用底层的内存只要有可能),因此你只能将数据复制到的实际的流时你知道它会适应。

Job done. This avoids writing partial objects by completing each individual object to the memory-stream (reusing the underlying memory whenever possible), and thus you only copy the data to the actual stream when you know it will fit.

下面是一个使用一个完整的例子 protobuf网:

Here's a complete example using protobuf-net:

using System;
using System.Collections.Generic;
using System.IO;
using System.Xml.Serialization;
using ProtoBuf;
[XmlType]      // not actually serialized as xml; simply it needs
class MyData   // a way of identifying the members, and it supports
{              // the Order property via XmlType/XmlElement
    [XmlElement(Order = 1)] public int Id { get; set; }
    [XmlElement(Order = 2)] public string Name { get; set; }
}
static class Program
{
    static IEnumerable<MyData> GetItems()
    {
        Random rand = new Random();
        int count = 0;
        while (true) // an infinite sequence of data; mwahahaahah
        {
            yield return new MyData
            {
                Id = rand.Next(0, 5000),
                Name = "Item " + count++
            };
        }
    }
    static void Main()
    {
        int space = 2048, count = 0;
        long checksum = 0;
        using(Stream dest = File.Create("out.bin"))
        using(MemoryStream buffer = new MemoryStream())
        {
            foreach (MyData obj in GetItems())
            {
                buffer.SetLength(0); // reset to empty, but retaining buffer to reduce allocs
                Serializer.SerializeWithLengthPrefix(buffer, obj, PrefixStyle.Base128, 1);
                int len = (int)buffer.Length;
                if(buffer.Length <= space) {
                    // add our item
                    dest.Write(buffer.GetBuffer(), 0, len);
                    space -= len;
                    checksum += obj.Id;
                    count++;
                } else {
                    break; // or new file, whatever
                }
            }
        }
        Console.WriteLine("Wrote " + count + " objects; chk = " + checksum);

        using (Stream source = File.OpenRead("out.bin"))
        {
            count = 0;
            checksum = 0;
            foreach (MyData item in
                Serializer.DeserializeItems<MyData>(source, PrefixStyle.Base128, 1))
            {
                count++;
                checksum += item.Id;
            }
        }

        Console.WriteLine("Read " + count + " objects; chk = " + checksum);
    }
}

您的可以的能够扩展这种方法与其他串行(如的BinaryFormatter 的DataContractSerializer ),但你presumably必须写自己的长度 - preFIX(在写),消耗它(在读),并限制数据。不是火箭科学,但并非易如反掌。

You may be able to extend this approach to other serializers (such as BinaryFormatter or DataContractSerializer), but you'll presumably have to write your own length-prefix (at write), consume it (at read) and limit the data. Not rocket science, but not entirely trivial.