Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kiBNe-0002y0-CQ for pgsql-hackers@arkaria.postgresql.org; Thu, 26 Nov 2020 07:07:50 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1kiBNc-00032E-Oj for pgsql-hackers@arkaria.postgresql.org; Thu, 26 Nov 2020 07:07:48 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kiBNc-000327-8G for pgsql-hackers@lists.postgresql.org; Thu, 26 Nov 2020 07:07:48 +0000 Received: from oss.nttdata.com ([49.212.34.109]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kiBNU-0004qm-Iq for pgsql-hackers@postgresql.org; Thu, 26 Nov 2020 07:07:47 +0000 Received: from oss.nttdata.com (localhost [127.0.0.1]) by oss.nttdata.com (Postfix) with ESMTP id 4D75F626EF; Thu, 26 Nov 2020 16:07:37 +0900 (JST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.102.3 at oss.nttdata.com MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=_abe83e456b3b0229d41ec343fd87da92" Date: Thu, 26 Nov 2020 16:07:37 +0900 From: Masahiro Ikeda To: Fujii Masao Cc: pgsql-hackers Subject: Re: Add statistics to pg_stat_wal view for wal related parameter tuning In-Reply-To: <58ccc451-8429-ce44-f0b7-773c8dfedf45@oss.nttdata.com> References: <35ef960128b90bfae3b3fdf60a3a860f@oss.nttdata.com> <11e81f0b7d90dd69077d3d964a9a4da9@oss.nttdata.com> <79620b5c-ecf1-3d4f-80b5-7e311b7a0662@oss.nttdata.com> <89de5056627ebdd144967959bd789bd6@oss.nttdata.com> <58ccc451-8429-ce44-f0b7-773c8dfedf45@oss.nttdata.com> Message-ID: <7eb4ebd487b53200bde98ba2f0a7e789@oss.nttdata.com> X-Sender: ikedamsh@oss.nttdata.com User-Agent: Roundcube Webmail/1.1.9 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --=_abe83e456b3b0229d41ec343fd87da92 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8; format=flowed On 2020-11-25 20:19, Fujii Masao wrote: > On 2020/11/19 16:31, Masahiro Ikeda wrote: >> On 2020-11-17 11:46, Fujii Masao wrote: >>> On 2020/11/16 16:35, Masahiro Ikeda wrote: >>>> On 2020-11-12 14:58, Fujii Masao wrote: >>>>> On 2020/11/06 10:25, Masahiro Ikeda wrote: >>>>>> On 2020-10-30 11:50, Fujii Masao wrote: >>>>>>> On 2020/10/29 17:03, Masahiro Ikeda wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> Thanks for your comments and advice. I updated the patch. >>>>>>>> >>>>>>>> On 2020-10-21 18:03, Kyotaro Horiguchi wrote: >>>>>>>>> At Tue, 20 Oct 2020 16:11:29 +0900, Masahiro Ikeda >>>>>>>>> wrote in >>>>>>>>>> On 2020-10-20 12:46, Amit Kapila wrote: >>>>>>>>>> > I see that we also need to add extra code to capture these stats (some >>>>>>>>>> > of which is in performance-critical path especially in >>>>>>>>>> > XLogInsertRecord) which again makes me a bit uncomfortable. It might >>>>>>>>>> > be that it is all fine as it is very important to collect these stats >>>>>>>>>> > at cluster-level in spite that the same information can be gathered at >>>>>>>>>> > statement-level to help customers but I don't see a very strong case >>>>>>>>>> > for that in your proposal. >>>>>>>>> >>>>>>>>> We should avoid that duplication as possible even if the both >>>>>>>>> number >>>>>>>>> are important. >>>>>>>>> >>>>>>>>>> Also about performance, I thought there are few impacts >>>>>>>>>> because it >>>>>>>>>> increments stats in memory. If I can implement to reuse >>>>>>>>>> pgWalUsage's >>>>>>>>>> value which already collects these stats, there is no impact >>>>>>>>>> in >>>>>>>>>> XLogInsertRecord. >>>>>>>>>> For example, how about pg_stat_wal() calculates the >>>>>>>>>> accumulated >>>>>>>>>> value of wal_records, wal_fpi, and wal_bytes to use >>>>>>>>>> pgWalUsage's >>>>>>>>>> value? >>>>>>>>> >>>>>>>>> I don't think that works, but it would work that >>>>>>>>> pgstat_send_wal() >>>>>>>>> takes the difference of that values between two successive >>>>>>>>> calls. >>>>>>>>> >>>>>>>>> WalUsage prevWalUsage; >>>>>>>>> ... >>>>>>>>> pgstat_send_wal() >>>>>>>>> { >>>>>>>>> .. >>>>>>>>>    /* fill in some values using pgWalUsage */ >>>>>>>>>    WalStats.m_wal_bytes   = pgWalUsage.wal_bytes   - >>>>>>>>> prevWalUsage.wal_bytes; >>>>>>>>>    WalStats.m_wal_records = pgWalUsage.wal_records - >>>>>>>>> prevWalUsage.wal_records; >>>>>>>>>    WalStats.m_wal_wal_fpi = pgWalUsage.wal_fpi     - >>>>>>>>> prevWalUsage.wal_fpi; >>>>>>>>> ... >>>>>>>>>    pgstat_send(&WalStats, sizeof(WalStats)); >>>>>>>>> >>>>>>>>>    /* remember the current numbers */ >>>>>>>>>    prevWalUsage = pgWalUsage; >>>>>>>> >>>>>>>> Thanks for Horiguchi-san's advice, I changed to reuse pgWalUsage >>>>>>>> which is already defined and eliminates the extra overhead. >>>>>>> >>>>>>> +    /* fill in some values using pgWalUsage */ >>>>>>> +    WalStats.m_wal_bytes = pgWalUsage.wal_bytes - >>>>>>> prevWalUsage.wal_bytes; >>>>>>> +    WalStats.m_wal_records = pgWalUsage.wal_records - >>>>>>> prevWalUsage.wal_records; >>>>>>> +    WalStats.m_wal_fpi = pgWalUsage.wal_fpi - >>>>>>> prevWalUsage.wal_fpi; >>>>>>> >>>>>>> It's better to use WalUsageAccumDiff() here? >>>>>> >>>>>> Yes, thanks. I fixed it. >>>>>> >>>>>>> prevWalUsage needs to be initialized with pgWalUsage? >>>>>>> >>>>>>> +                if (AmWalWriterProcess()){ >>>>>>> +                    WalStats.m_wal_write_walwriter++; >>>>>>> +                } >>>>>>> +                else >>>>>>> +                { >>>>>>> +                    WalStats.m_wal_write_backend++; >>>>>>> +                } >>>>>>> >>>>>>> I think that it's better not to separate m_wal_write_xxx into two >>>>>>> for >>>>>>> walwriter and other processes. Instead, we can use one >>>>>>> m_wal_write_xxx >>>>>>> counter and make pgstat_send_wal() send also the process type to >>>>>>> the stats collector. Then the stats collector can accumulate the >>>>>>> counters >>>>>>> per process type if necessary. If we adopt this approach, we can >>>>>>> easily >>>>>>> extend pg_stat_wal so that any fields can be reported per process >>>>>>> type. >>>>>> >>>>>> I'll remove the above source code because these counters are not >>>>>> useful. >>>>>> >>>>>> >>>>>> On 2020-10-30 12:00, Fujii Masao wrote: >>>>>>> On 2020/10/20 11:31, Masahiro Ikeda wrote: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I think we need to add some statistics to pg_stat_wal view. >>>>>>>> >>>>>>>> Although there are some parameter related WAL, >>>>>>>> there are few statistics for tuning them. >>>>>>>> >>>>>>>> I think it's better to provide the following statistics. >>>>>>>> Please let me know your comments. >>>>>>>> >>>>>>>> ``` >>>>>>>> postgres=# SELECT * from pg_stat_wal; >>>>>>>> -[ RECORD 1 ]-------+------------------------------ >>>>>>>> wal_records         | 2000224 >>>>>>>> wal_fpi             | 47 >>>>>>>> wal_bytes           | 248216337 >>>>>>>> wal_buffers_full    | 20954 >>>>>>>> wal_init_file       | 8 >>>>>>>> wal_write_backend   | 20960 >>>>>>>> wal_write_walwriter | 46 >>>>>>>> wal_write_time      | 51 >>>>>>>> wal_sync_backend    | 7 >>>>>>>> wal_sync_walwriter  | 8 >>>>>>>> wal_sync_time       | 0 >>>>>>>> stats_reset         | 2020-10-20 11:04:51.307771+09 >>>>>>>> ``` >>>>>>>> >>>>>>>> 1. Basic statistics of WAL activity >>>>>>>> >>>>>>>> - wal_records: Total number of WAL records generated >>>>>>>> - wal_fpi: Total number of WAL full page images generated >>>>>>>> - wal_bytes: Total amount of WAL bytes generated >>>>>>>> >>>>>>>> To understand DB's performance, first, we will check the >>>>>>>> performance >>>>>>>> trends for the entire database instance. >>>>>>>> For example, if the number of wal_fpi becomes higher, users may >>>>>>>> tune >>>>>>>> "wal_compression", "checkpoint_timeout" and so on. >>>>>>>> >>>>>>>> Although users can check the above statistics via EXPLAIN, >>>>>>>> auto_explain, >>>>>>>> autovacuum and pg_stat_statements now, >>>>>>>> if users want to see the performance trends  for the entire >>>>>>>> database, >>>>>>>> they must recalculate the statistics. >>>>>>>> >>>>>>>> I think it is useful to add the sum of the basic statistics. >>>>>>>> >>>>>>>> >>>>>>>> 2.  WAL segment file creation >>>>>>>> >>>>>>>> - wal_init_file: Total number of WAL segment files created. >>>>>>>> >>>>>>>> To create a new WAL file may have an impact on the performance >>>>>>>> of >>>>>>>> a write-heavy workload generating lots of WAL. If this number is >>>>>>>> reported high, >>>>>>>> to reduce the number of this initialization, we can tune >>>>>>>> WAL-related parameters >>>>>>>> so that more "recycled" WAL files can be held. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> 3. Number of when WAL is flushed >>>>>>>> >>>>>>>> - wal_write_backend : Total number of WAL data written to the >>>>>>>> disk by backends >>>>>>>> - wal_write_walwriter : Total number of WAL data written to the >>>>>>>> disk by walwriter >>>>>>>> - wal_sync_backend : Total number of WAL data synced to the disk >>>>>>>> by backends >>>>>>>> - wal_sync_walwriter : Total number of WAL data synced to the >>>>>>>> disk by walwrite >>>>>>>> >>>>>>>> I think it's useful for tuning "synchronous_commit" and >>>>>>>> "commit_delay" for query executions. >>>>>>>> If the number of WAL is flushed is high, users can know >>>>>>>> "synchronous_commit" is useful for the workload. >>>>>>> >>>>>>> I just wonder how useful these counters are. Even without these >>>>>>> counters, >>>>>>> we already know synchronous_commit=off is likely to cause the >>>>>>> better >>>>>>> performance (but has the risk of data loss). So ISTM that these >>>>>>> counters are >>>>>>> not so useful when tuning synchronous_commit. >>>>>> >>>>>> Thanks, my understanding was wrong. >>>>>> I agreed that your comments. >>>>>> >>>>>> I merged the statistics of *_backend and *_walwriter. >>>>>> I think the sum of them is useful to calculate the average per >>>>>> write/sync time. >>>>>> For example, per write time is equals wal_write_time / wal_write. >>>>> >>>>> Understood. >>>>> >>>>> Thanks for updating the patch! >>>> >>>> Thanks for your comments. >>>> >>>>> patching file src/include/catalog/pg_proc.dat >>>>> Hunk #1 FAILED at 5491. >>>>> 1 out of 1 hunk FAILED -- saving rejects to file >>>>> src/include/catalog/pg_proc.dat.rej >>>>> >>>>> I got this failure when applying the patch. Could you update the >>>>> patch? >>>> >>>> Thanks, I updated the patch. >>>> >>>>> -       Number of times WAL data was written to the disk because >>>>> WAL >>>>> buffers got full >>>>> +       Total number of times WAL data written to the disk because >>>>> WAL >>>>> buffers got full >>>>> >>>>> Isn't "was" necessary between "data" and "written"? >>>> >>>> Yes, I fixed it. >>>> >>>>> +      >>>> role="column_definition"> >>>>> +       wal_bytes bigint >>>>> >>>>> Shouldn't the type of wal_bytes be numeric because the total number >>>>> of >>>>> WAL bytes can exceed the range of bigint? I think that the type of >>>>> pg_stat_statements.wal_bytes is also numeric for the same reason. >>>> >>>> Thanks, I fixed it. >>>> >>>> Since I cast the type of wal_bytes from PgStat_Counter to uint64, >>>> I changed the type of PgStat_MsgWal and PgStat_WalStats too. >>>> >>>>> +      >>>> role="column_definition"> >>>>> +       wal_write_time >>>>> bigint >>>>> >>>>> Shouldn't the type of wal_xxx_time be double precision, >>>>> like pg_stat_database.blk_write_time? >>>> >>>> Thanks, I changed it. >>>> >>>>> Even when fsync is set to off or wal_sync_method is set to >>>>> open_sync, >>>>> wal_sync is incremented. Isn't this behavior confusing? >>> >>> What do you think about this comment? >> >> Sorry, I'll change to increment wal_sync and wal_sync_time only >> if a specific fsync method is called. >> >>> I found that we discussed track-WAL-IO-timing feature at the past >>> discussion >>> about the similar feature [1]. But the feature was droppped from the >>> proposal >>> patch because there was the performance concern. So probably we need >>> to >>> revisit the past discussion and benchmark the performance. Thought? >>> >>> If track-WAL-IO-timing feature may cause performance regression, >>> it might be an idea to extract wal_records, wal_fpi and wal_bytes >>> parts >>> from the patch and commit it at first. >>> >>> [1] >>> https://postgr.es/m/CAJrrPGc6APFUGYNcPe4qcNxpL8gXKYv1KST+vwJcFtCSCEySnA@mail.gmail.com >> >> Thanks, I'll check the thread. >> I agree to add basic statistics at first and I attached the patch. > > Thanks! > > + /* Send WAL statistics */ > + pgstat_send_wal(); > > This is not necessary because walwriter generates no WAL data? No, it's not necessary. Thanks. I fixed it. Regards, -- Masahiro Ikeda NTT DATA CORPORATION --=_abe83e456b3b0229d41ec343fd87da92 Content-Transfer-Encoding: base64 Content-Type: text/x-diff; name=0002_add_basic_statistics_to_pg_stat_wal_view.patch Content-Disposition: attachment; filename=0002_add_basic_statistics_to_pg_stat_wal_view.patch; size=8448 ZGlmZiAtLWdpdCBhL2RvYy9zcmMvc2dtbC9tb25pdG9yaW5nLnNnbWwgYi9kb2Mvc3JjL3NnbWwv bW9uaXRvcmluZy5zZ21sCmluZGV4IDk4ZTE5OTU0NTMuLjZlNjYxZDZmYWUgMTAwNjQ0Ci0tLSBh L2RvYy9zcmMvc2dtbC9tb25pdG9yaW5nLnNnbWwKKysrIGIvZG9jL3NyYy9zZ21sL21vbml0b3Jp bmcuc2dtbApAQCAtMzQ0Nyw2ICszNDQ3LDMzIEBAIFNFTEVDVCBwaWQsIHdhaXRfZXZlbnRfdHlw ZSwgd2FpdF9ldmVudCBGUk9NIHBnX3N0YXRfYWN0aXZpdHkgV0hFUkUgd2FpdF9ldmVudCBpCiAg ICAgPC90aGVhZD4KIAogICAgIDx0Ym9keT4KKyAgICAgPHJvdz4KKyAgICAgIDxlbnRyeSByb2xl PSJjYXRhbG9nX3RhYmxlX2VudHJ5Ij48cGFyYSByb2xlPSJjb2x1bW5fZGVmaW5pdGlvbiI+Cisg ICAgICAgPHN0cnVjdGZpZWxkPndhbF9yZWNvcmRzPC9zdHJ1Y3RmaWVsZD4gPHR5cGU+YmlnaW50 PC90eXBlPgorICAgICAgPC9wYXJhPgorICAgICAgPHBhcmE+CisgICAgICAgVG90YWwgbnVtYmVy IG9mIFdBTCByZWNvcmRzIGdlbmVyYXRlZAorICAgICAgPC9wYXJhPjwvZW50cnk+CisgICAgIDwv cm93PgorCisgICAgIDxyb3c+CisgICAgICA8ZW50cnkgcm9sZT0iY2F0YWxvZ190YWJsZV9lbnRy eSI+PHBhcmEgcm9sZT0iY29sdW1uX2RlZmluaXRpb24iPgorICAgICAgIDxzdHJ1Y3RmaWVsZD53 YWxfZnBpPC9zdHJ1Y3RmaWVsZD4gPHR5cGU+YmlnaW50PC90eXBlPgorICAgICAgPC9wYXJhPgor ICAgICAgPHBhcmE+CisgICAgICAgVG90YWwgbnVtYmVyIG9mIFdBTCBmdWxsIHBhZ2UgaW1hZ2Vz IGdlbmVyYXRlZAorICAgICAgPC9wYXJhPjwvZW50cnk+CisgICAgIDwvcm93PgorCisgICAgIDxy b3c+CisgICAgICA8ZW50cnkgcm9sZT0iY2F0YWxvZ190YWJsZV9lbnRyeSI+PHBhcmEgcm9sZT0i Y29sdW1uX2RlZmluaXRpb24iPgorICAgICAgIDxzdHJ1Y3RmaWVsZD53YWxfYnl0ZXM8L3N0cnVj dGZpZWxkPiA8dHlwZT5udW1lcmljPC90eXBlPgorICAgICAgPC9wYXJhPgorICAgICAgPHBhcmE+ CisgICAgICAgVG90YWwgYW1vdW50IG9mIFdBTCBieXRlcyBnZW5lcmF0ZWQKKyAgICAgIDwvcGFy YT48L2VudHJ5PgorICAgICA8L3Jvdz4KKwogICAgICA8cm93PgogICAgICAgPGVudHJ5IHJvbGU9 ImNhdGFsb2dfdGFibGVfZW50cnkiPjxwYXJhIHJvbGU9ImNvbHVtbl9kZWZpbml0aW9uIj4KICAg ICAgICA8c3RydWN0ZmllbGQ+d2FsX2J1ZmZlcnNfZnVsbDwvc3RydWN0ZmllbGQ+IDx0eXBlPmJp Z2ludDwvdHlwZT4KZGlmZiAtLWdpdCBhL3NyYy9iYWNrZW5kL2NhdGFsb2cvc3lzdGVtX3ZpZXdz LnNxbCBiL3NyYy9iYWNrZW5kL2NhdGFsb2cvc3lzdGVtX3ZpZXdzLnNxbAppbmRleCAyZTRhYTFj NGI2Li5iMTQwYzIxMGJjIDEwMDY0NAotLS0gYS9zcmMvYmFja2VuZC9jYXRhbG9nL3N5c3RlbV92 aWV3cy5zcWwKKysrIGIvc3JjL2JhY2tlbmQvY2F0YWxvZy9zeXN0ZW1fdmlld3Muc3FsCkBAIC05 OTMsNiArOTkzLDkgQEAgQ1JFQVRFIFZJRVcgcGdfc3RhdF9iZ3dyaXRlciBBUwogCiBDUkVBVEUg VklFVyBwZ19zdGF0X3dhbCBBUwogICAgIFNFTEVDVAorICAgICAgICB3LndhbF9yZWNvcmRzLAor ICAgICAgICB3LndhbF9mcGksCisgICAgICAgIHcud2FsX2J5dGVzLAogICAgICAgICB3LndhbF9i dWZmZXJzX2Z1bGwsCiAgICAgICAgIHcuc3RhdHNfcmVzZXQKICAgICBGUk9NIHBnX3N0YXRfZ2V0 X3dhbCgpIHc7CmRpZmYgLS1naXQgYS9zcmMvYmFja2VuZC9wb3N0bWFzdGVyL3Bnc3RhdC5jIGIv c3JjL2JhY2tlbmQvcG9zdG1hc3Rlci9wZ3N0YXQuYwppbmRleCBlNzZlNjI3YzZiLi43ODFmMmE1 MjQ4IDEwMDY0NAotLS0gYS9zcmMvYmFja2VuZC9wb3N0bWFzdGVyL3Bnc3RhdC5jCisrKyBiL3Ny Yy9iYWNrZW5kL3Bvc3RtYXN0ZXIvcGdzdGF0LmMKQEAgLTQxLDYgKzQxLDcgQEAKICNpbmNsdWRl ICJjYXRhbG9nL3BnX2RhdGFiYXNlLmgiCiAjaW5jbHVkZSAiY2F0YWxvZy9wZ19wcm9jLmgiCiAj aW5jbHVkZSAiY29tbW9uL2lwLmgiCisjaW5jbHVkZSAiZXhlY3V0b3IvaW5zdHJ1bWVudC5oIgog I2luY2x1ZGUgImxpYnBxL2xpYnBxLmgiCiAjaW5jbHVkZSAibGlicHEvcHFzaWduYWwuaCIKICNp bmNsdWRlICJtYi9wZ193Y2hhci5oIgpAQCAtMTQyLDYgKzE0Myw3IEBAIGNoYXIJICAgKnBnc3Rh dF9zdGF0X3RtcG5hbWUgPSBOVUxMOwogICovCiBQZ1N0YXRfTXNnQmdXcml0ZXIgQmdXcml0ZXJT dGF0czsKIFBnU3RhdF9Nc2dXYWwgV2FsU3RhdHM7CitzdGF0aWMgV2FsVXNhZ2UgcHJldldhbFVz YWdlOwogCiAvKgogICogTGlzdCBvZiBTTFJVIG5hbWVzIHRoYXQgd2Uga2VlcCBzdGF0cyBmb3Iu ICBUaGVyZSBpcyBubyBjZW50cmFsIHJlZ2lzdHJ5IG9mCkBAIC0zMDQ4LDYgKzMwNTAsOSBAQCBw Z3N0YXRfaW5pdGlhbGl6ZSh2b2lkKQogCQlNeUJFRW50cnkgPSAmQmFja2VuZFN0YXR1c0FycmF5 W01heEJhY2tlbmRzICsgTXlBdXhQcm9jVHlwZV07CiAJfQogCisJLyogSW5pdGlhbGl6ZSBXQUwg dXNhZ2Ugc3RhdHVzIGluZm9ybWF0aW9uICovCisJcHJldldhbFVzYWdlID0gcGdXYWxVc2FnZTsK KwogCS8qIFNldCB1cCBhIHByb2Nlc3MtZXhpdCBob29rIHRvIGNsZWFuIHVwICovCiAJb25fc2ht ZW1fZXhpdChwZ3N0YXRfYmVzaHV0ZG93bl9ob29rLCAwKTsKIH0KQEAgLTQ1NzcsNiArNDU4Miwx NSBAQCBwZ3N0YXRfc2VuZF93YWwodm9pZCkKIAkvKiBXZSBhc3N1bWUgdGhpcyBpbml0aWFsaXpl cyB0byB6ZXJvZXMgKi8KIAlzdGF0aWMgY29uc3QgUGdTdGF0X01zZ1dhbCBhbGxfemVyb2VzOwog CisJLyogZmlsbCBpbiBzb21lIHZhbHVlcyB1c2luZyBwZ1dhbFVzYWdlICovCisJV2FsVXNhZ2Ug d2FsdXNhZ2U7CisJbWVtc2V0KCZ3YWx1c2FnZSwgMCwgc2l6ZW9mKFdhbFVzYWdlKSk7CisJV2Fs VXNhZ2VBY2N1bURpZmYoJndhbHVzYWdlLCAmcGdXYWxVc2FnZSwgJnByZXZXYWxVc2FnZSk7CisK KwlXYWxTdGF0cy5tX3dhbF9ieXRlcyA9IHdhbHVzYWdlLndhbF9ieXRlczsKKwlXYWxTdGF0cy5t X3dhbF9yZWNvcmRzID0gd2FsdXNhZ2Uud2FsX3JlY29yZHM7CisJV2FsU3RhdHMubV93YWxfZnBp ID0gd2FsdXNhZ2Uud2FsX2ZwaTsKKwogCS8qCiAJICogVGhpcyBmdW5jdGlvbiBjYW4gYmUgY2Fs bGVkIGV2ZW4gaWYgbm90aGluZyBhdCBhbGwgaGFzIGhhcHBlbmVkLiBJbgogCSAqIHRoaXMgY2Fz ZSwgYXZvaWQgc2VuZGluZyBhIGNvbXBsZXRlbHkgZW1wdHkgbWVzc2FnZSB0byB0aGUgc3RhdHMK QEAgLTQ1OTEsNiArNDYwNSw5IEBAIHBnc3RhdF9zZW5kX3dhbCh2b2lkKQogCXBnc3RhdF9zZXRo ZWFkZXIoJldhbFN0YXRzLm1faGRyLCBQR1NUQVRfTVRZUEVfV0FMKTsKIAlwZ3N0YXRfc2VuZCgm V2FsU3RhdHMsIHNpemVvZihXYWxTdGF0cykpOwogCisJLyogcmVtZW1iZXIgdGhlIGN1cnJlbnQg bnVtYmVycyAqLworCXByZXZXYWxVc2FnZSA9IHBnV2FsVXNhZ2U7CisKIAkvKgogCSAqIENsZWFy IG91dCB0aGUgc3RhdGlzdGljcyBidWZmZXIsIHNvIGl0IGNhbiBiZSByZS11c2VkLgogCSAqLwpA QCAtNjc1OSw2ICs2Nzc2LDkgQEAgcGdzdGF0X3JlY3ZfYmd3cml0ZXIoUGdTdGF0X01zZ0JnV3Jp dGVyICptc2csIGludCBsZW4pCiBzdGF0aWMgdm9pZAogcGdzdGF0X3JlY3Zfd2FsKFBnU3RhdF9N c2dXYWwgKm1zZywgaW50IGxlbikKIHsKKwl3YWxTdGF0cy53YWxfcmVjb3JkcyArPSBtc2ctPm1f d2FsX3JlY29yZHM7CisJd2FsU3RhdHMud2FsX2ZwaSArPSBtc2ctPm1fd2FsX2ZwaTsKKwl3YWxT dGF0cy53YWxfYnl0ZXMgKz0gbXNnLT5tX3dhbF9ieXRlczsKIAl3YWxTdGF0cy53YWxfYnVmZmVy c19mdWxsICs9IG1zZy0+bV93YWxfYnVmZmVyc19mdWxsOwogfQogCmRpZmYgLS1naXQgYS9zcmMv YmFja2VuZC91dGlscy9hZHQvcGdzdGF0ZnVuY3MuYyBiL3NyYy9iYWNrZW5kL3V0aWxzL2FkdC9w Z3N0YXRmdW5jcy5jCmluZGV4IGEyMTBmYzkzYjQuLjZhZmUxYjZmNTYgMTAwNjQ0Ci0tLSBhL3Ny Yy9iYWNrZW5kL3V0aWxzL2FkdC9wZ3N0YXRmdW5jcy5jCisrKyBiL3NyYy9iYWNrZW5kL3V0aWxz L2FkdC9wZ3N0YXRmdW5jcy5jCkBAIC0xNzAzLDEwICsxNzAzLDExIEBAIHBnX3N0YXRfZ2V0X2J1 Zl9hbGxvYyhQR19GVU5DVElPTl9BUkdTKQogRGF0dW0KIHBnX3N0YXRfZ2V0X3dhbChQR19GVU5D VElPTl9BUkdTKQogewotI2RlZmluZSBQR19TVEFUX0dFVF9XQUxfQ09MUwkyCisjZGVmaW5lIFBH X1NUQVRfR0VUX1dBTF9DT0xTCTUKIAlUdXBsZURlc2MJdHVwZGVzYzsKIAlEYXR1bQkJdmFsdWVz W1BHX1NUQVRfR0VUX1dBTF9DT0xTXTsKIAlib29sCQludWxsc1tQR19TVEFUX0dFVF9XQUxfQ09M U107CisJY2hhcgkJYnVmWzI1Nl07CiAJUGdTdGF0X1dhbFN0YXRzICp3YWxfc3RhdHM7CiAKIAkv KiBJbml0aWFsaXNlIHZhbHVlcyBhbmQgTlVMTCBmbGFncyBhcnJheXMgKi8KQEAgLTE3MTUsOSAr MTcxNiwxNSBAQCBwZ19zdGF0X2dldF93YWwoUEdfRlVOQ1RJT05fQVJHUykKIAogCS8qIEluaXRp YWxpc2UgYXR0cmlidXRlcyBpbmZvcm1hdGlvbiBpbiB0aGUgdHVwbGUgZGVzY3JpcHRvciAqLwog CXR1cGRlc2MgPSBDcmVhdGVUZW1wbGF0ZVR1cGxlRGVzYyhQR19TVEFUX0dFVF9XQUxfQ09MUyk7 Ci0JVHVwbGVEZXNjSW5pdEVudHJ5KHR1cGRlc2MsIChBdHRyTnVtYmVyKSAxLCAid2FsX2J1ZmZl cnNfZnVsbCIsCisJVHVwbGVEZXNjSW5pdEVudHJ5KHR1cGRlc2MsIChBdHRyTnVtYmVyKSAxLCAi d2FsX3JlY29yZHMiLAogCQkJCQkgICBJTlQ4T0lELCAtMSwgMCk7Ci0JVHVwbGVEZXNjSW5pdEVu dHJ5KHR1cGRlc2MsIChBdHRyTnVtYmVyKSAyLCAic3RhdHNfcmVzZXQiLAorCVR1cGxlRGVzY0lu aXRFbnRyeSh0dXBkZXNjLCAoQXR0ck51bWJlcikgMiwgIndhbF9mcGkiLAorCQkJCQkgICBJTlQ4 T0lELCAtMSwgMCk7CisJVHVwbGVEZXNjSW5pdEVudHJ5KHR1cGRlc2MsIChBdHRyTnVtYmVyKSAz LCAid2FsX2J5dGVzIiwKKwkJCQkJICAgTlVNRVJJQ09JRCwgLTEsIDApOworCVR1cGxlRGVzY0lu aXRFbnRyeSh0dXBkZXNjLCAoQXR0ck51bWJlcikgNCwgIndhbF9idWZmZXJzX2Z1bGwiLAorCQkJ CQkgICBJTlQ4T0lELCAtMSwgMCk7CisJVHVwbGVEZXNjSW5pdEVudHJ5KHR1cGRlc2MsIChBdHRy TnVtYmVyKSA1LCAic3RhdHNfcmVzZXQiLAogCQkJCQkgICBUSU1FU1RBTVBUWk9JRCwgLTEsIDAp OwogCiAJQmxlc3NUdXBsZURlc2ModHVwZGVzYyk7CkBAIC0xNzI2LDggKzE3MzMsMTggQEAgcGdf c3RhdF9nZXRfd2FsKFBHX0ZVTkNUSU9OX0FSR1MpCiAJd2FsX3N0YXRzID0gcGdzdGF0X2ZldGNo X3N0YXRfd2FsKCk7CiAKIAkvKiBGaWxsIHZhbHVlcyBhbmQgTlVMTHMgKi8KLQl2YWx1ZXNbMF0g PSBJbnQ2NEdldERhdHVtKHdhbF9zdGF0cy0+d2FsX2J1ZmZlcnNfZnVsbCk7Ci0JdmFsdWVzWzFd ID0gVGltZXN0YW1wVHpHZXREYXR1bSh3YWxfc3RhdHMtPnN0YXRfcmVzZXRfdGltZXN0YW1wKTsK Kwl2YWx1ZXNbMF0gPSBJbnQ2NEdldERhdHVtKHdhbF9zdGF0cy0+d2FsX3JlY29yZHMpOworCXZh bHVlc1sxXSA9IEludDY0R2V0RGF0dW0od2FsX3N0YXRzLT53YWxfZnBpKTsKKworCS8qIENvbnZl cnQgdG8gbnVtZXJpYy4gKi8KKwlzbnByaW50ZihidWYsIHNpemVvZiBidWYsIFVJTlQ2NF9GT1JN QVQsIHdhbF9zdGF0cy0+d2FsX2J5dGVzKTsKKwl2YWx1ZXNbMl0gPSBEaXJlY3RGdW5jdGlvbkNh bGwzKG51bWVyaWNfaW4sCisJCQkJCQkJCQlDU3RyaW5nR2V0RGF0dW0oYnVmKSwKKwkJCQkJCQkJ CU9iamVjdElkR2V0RGF0dW0oMCksCisJCQkJCQkJCQlJbnQzMkdldERhdHVtKC0xKSk7CisKKwl2 YWx1ZXNbM10gPSBJbnQ2NEdldERhdHVtKHdhbF9zdGF0cy0+d2FsX2J1ZmZlcnNfZnVsbCk7CisJ dmFsdWVzWzRdID0gVGltZXN0YW1wVHpHZXREYXR1bSh3YWxfc3RhdHMtPnN0YXRfcmVzZXRfdGlt ZXN0YW1wKTsKIAogCS8qIFJldHVybnMgdGhlIHJlY29yZCBhcyBEYXR1bSAqLwogCVBHX1JFVFVS Tl9EQVRVTShIZWFwVHVwbGVHZXREYXR1bShoZWFwX2Zvcm1fdHVwbGUodHVwZGVzYywgdmFsdWVz LCBudWxscykpKTsKZGlmZiAtLWdpdCBhL3NyYy9pbmNsdWRlL2NhdGFsb2cvcGdfcHJvYy5kYXQg Yi9zcmMvaW5jbHVkZS9jYXRhbG9nL3BnX3Byb2MuZGF0CmluZGV4IGMwMWRhNGJmMDEuLjY3YWI4 NjEyNjAgMTAwNjQ0Ci0tLSBhL3NyYy9pbmNsdWRlL2NhdGFsb2cvcGdfcHJvYy5kYXQKKysrIGIv c3JjL2luY2x1ZGUvY2F0YWxvZy9wZ19wcm9jLmRhdApAQCAtNTUwMyw4ICs1NTAzLDkgQEAKIHsg b2lkID0+ICcxMTM2JywgZGVzY3IgPT4gJ3N0YXRpc3RpY3M6IGluZm9ybWF0aW9uIGFib3V0IFdB TCBhY3Rpdml0eScsCiAgIHByb25hbWUgPT4gJ3BnX3N0YXRfZ2V0X3dhbCcsIHByb2lzc3RyaWN0 ID0+ICdmJywgcHJvdm9sYXRpbGUgPT4gJ3MnLAogICBwcm9wYXJhbGxlbCA9PiAncicsIHByb3Jl dHR5cGUgPT4gJ3JlY29yZCcsIHByb2FyZ3R5cGVzID0+ICcnLAotICBwcm9hbGxhcmd0eXBlcyA9 PiAne2ludDgsdGltZXN0YW1wdHp9JywgcHJvYXJnbW9kZXMgPT4gJ3tvLG99JywKLSAgcHJvYXJn bmFtZXMgPT4gJ3t3YWxfYnVmZmVyc19mdWxsLHN0YXRzX3Jlc2V0fScsCisgICBwcm9hbGxhcmd0 eXBlcyA9PiAne2ludDgsaW50OCxudW1lcmljLGludDgsdGltZXN0YW1wdHp9JywKKyAgIHByb2Fy Z21vZGVzID0+ICd7byxvLG8sbyxvfScsCisgICBwcm9hcmduYW1lcyA9PiAne3dhbF9yZWNvcmRz LHdhbF9mcGksd2FsX2J5dGVzLHdhbF9idWZmZXJzX2Z1bGwsc3RhdHNfcmVzZXR9JywKICAgcHJv c3JjID0+ICdwZ19zdGF0X2dldF93YWwnIH0sCiAKIHsgb2lkID0+ICcyMzA2JywgZGVzY3IgPT4g J3N0YXRpc3RpY3M6IGluZm9ybWF0aW9uIGFib3V0IFNMUlUgY2FjaGVzJywKZGlmZiAtLWdpdCBh L3NyYy9pbmNsdWRlL3Bnc3RhdC5oIGIvc3JjL2luY2x1ZGUvcGdzdGF0LmgKaW5kZXggMjU3ZTUx NWJmZS4uY2VlM2QyOGEyZCAxMDA2NDQKLS0tIGEvc3JjL2luY2x1ZGUvcGdzdGF0LmgKKysrIGIv c3JjL2luY2x1ZGUvcGdzdGF0LmgKQEAgLTQ1OSw2ICs0NTksOSBAQCB0eXBlZGVmIHN0cnVjdCBQ Z1N0YXRfTXNnQmdXcml0ZXIKIHR5cGVkZWYgc3RydWN0IFBnU3RhdF9Nc2dXYWwKIHsKIAlQZ1N0 YXRfTXNnSGRyIG1faGRyOworCVBnU3RhdF9Db3VudGVyIG1fd2FsX3JlY29yZHM7CisJUGdTdGF0 X0NvdW50ZXIgbV93YWxfZnBpOworCXVpbnQ2NCBtX3dhbF9ieXRlczsKIAlQZ1N0YXRfQ291bnRl ciBtX3dhbF9idWZmZXJzX2Z1bGw7CiB9IFBnU3RhdF9Nc2dXYWw7CiAKQEAgLTc5OCw2ICs4MDEs OSBAQCB0eXBlZGVmIHN0cnVjdCBQZ1N0YXRfR2xvYmFsU3RhdHMKICAqLwogdHlwZWRlZiBzdHJ1 Y3QgUGdTdGF0X1dhbFN0YXRzCiB7CisJUGdTdGF0X0NvdW50ZXIgd2FsX3JlY29yZHM7CisJUGdT dGF0X0NvdW50ZXIgd2FsX2ZwaTsKKwl1aW50NjQgd2FsX2J5dGVzOwogCVBnU3RhdF9Db3VudGVy IHdhbF9idWZmZXJzX2Z1bGw7CiAJVGltZXN0YW1wVHogc3RhdF9yZXNldF90aW1lc3RhbXA7CiB9 IFBnU3RhdF9XYWxTdGF0czsKZGlmZiAtLWdpdCBhL3NyYy90ZXN0L3JlZ3Jlc3MvZXhwZWN0ZWQv cnVsZXMub3V0IGIvc3JjL3Rlc3QvcmVncmVzcy9leHBlY3RlZC9ydWxlcy5vdXQKaW5kZXggMDk3 ZmY1ZDExMS4uNjI5M2FiNTdiYyAxMDA2NDQKLS0tIGEvc3JjL3Rlc3QvcmVncmVzcy9leHBlY3Rl ZC9ydWxlcy5vdXQKKysrIGIvc3JjL3Rlc3QvcmVncmVzcy9leHBlY3RlZC9ydWxlcy5vdXQKQEAg LTIxMzgsOSArMjEzOCwxMiBAQCBwZ19zdGF0X3VzZXJfdGFibGVzfCBTRUxFQ1QgcGdfc3RhdF9h bGxfdGFibGVzLnJlbGlkLAogICAgIHBnX3N0YXRfYWxsX3RhYmxlcy5hdXRvYW5hbHl6ZV9jb3Vu dAogICAgRlJPTSBwZ19zdGF0X2FsbF90YWJsZXMKICAgV0hFUkUgKChwZ19zdGF0X2FsbF90YWJs ZXMuc2NoZW1hbmFtZSA8PiBBTEwgKEFSUkFZWydwZ19jYXRhbG9nJzo6bmFtZSwgJ2luZm9ybWF0 aW9uX3NjaGVtYSc6Om5hbWVdKSkgQU5EIChwZ19zdGF0X2FsbF90YWJsZXMuc2NoZW1hbmFtZSAh fiAnXnBnX3RvYXN0Jzo6dGV4dCkpOwotcGdfc3RhdF93YWx8IFNFTEVDVCB3LndhbF9idWZmZXJz X2Z1bGwsCitwZ19zdGF0X3dhbHwgU0VMRUNUIHcud2FsX3JlY29yZHMsCisgICAgdy53YWxfZnBp LAorICAgIHcud2FsX2J5dGVzLAorICAgIHcud2FsX2J1ZmZlcnNfZnVsbCwKICAgICB3LnN0YXRz X3Jlc2V0Ci0gICBGUk9NIHBnX3N0YXRfZ2V0X3dhbCgpIHcod2FsX2J1ZmZlcnNfZnVsbCwgc3Rh dHNfcmVzZXQpOworICAgRlJPTSBwZ19zdGF0X2dldF93YWwoKSB3KHdhbF9yZWNvcmRzLCB3YWxf ZnBpLCB3YWxfYnl0ZXMsIHdhbF9idWZmZXJzX2Z1bGwsIHN0YXRzX3Jlc2V0KTsKIHBnX3N0YXRf d2FsX3JlY2VpdmVyfCBTRUxFQ1Qgcy5waWQsCiAgICAgcy5zdGF0dXMsCiAgICAgcy5yZWNlaXZl X3N0YXJ0X2xzbiwK --=_abe83e456b3b0229d41ec343fd87da92 Content-Transfer-Encoding: base64 Content-Type: text/plain; name=0001_0002.diff Content-Disposition: attachment; filename=0001_0002.diff; size=549 MTE5LDEzMmQxMTgKPCBkaWZmIC0tZ2l0IGEvc3JjL2JhY2tlbmQvcG9zdG1hc3Rlci93YWx3cml0 ZXIuYyBiL3NyYy9iYWNrZW5kL3Bvc3RtYXN0ZXIvd2Fsd3JpdGVyLmMKPCBpbmRleCBhNTI4MzJm ZTkwLi5jZTlmNGI3Y2Y3IDEwMDY0NAo8IC0tLSBhL3NyYy9iYWNrZW5kL3Bvc3RtYXN0ZXIvd2Fs d3JpdGVyLmMKPCArKysgYi9zcmMvYmFja2VuZC9wb3N0bWFzdGVyL3dhbHdyaXRlci5jCjwgQEAg LTI1Myw2ICsyNTMsOSBAQCBXYWxXcml0ZXJNYWluKHZvaWQpCjwgIAkJZWxzZSBpZiAobGVmdF90 aWxsX2hpYmVybmF0ZSA+IDApCjwgIAkJCWxlZnRfdGlsbF9oaWJlcm5hdGUtLTsKPCAgCjwgKwkJ LyogU2VuZCBXQUwgc3RhdGlzdGljcyAqLwo8ICsJCXBnc3RhdF9zZW5kX3dhbCgpOwo8ICsKPCAg CQkvKgo8ICAJCSAqIFNsZWVwIHVudGlsIHdlIGFyZSBzaWduYWxlZCBvciBXYWxXcml0ZXJEZWxh eSBoYXMgZWxhcHNlZC4gIElmIHdlCjwgIAkJICogaGF2ZW4ndCBkb25lIGFueXRoaW5nIHVzZWZ1 bCBmb3IgcXVpdGUgc29tZSB0aW1lLCBsZW5ndGhlbiB0aGUK --=_abe83e456b3b0229d41ec343fd87da92--