From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Justin Pryzby <pryzby@telsasoft.com>
To: Laurenz Albe <laurenz.albe@cybertec.at>
Cc: Dinesh Chandra 12108 <Dinesh.Chandra@cyient.com>
Cc: pgsql-performance@postgresql.org <pgsql-performance@postgresql.org>
Subject: Re: CPU 100% usage caused by unknown postgres process..
Date: Wed, 13 Dec 2017 16:22:47 +0100
Message-ID: <96e72d15-7edf-96bd-69e1-59ef562f4ca8@2ndquadrant.com> (raw)
In-Reply-To: <20171213121952.GA16691@telsasoft.com>
References: <68dd64b2a09f4fd8afe9582a4e23641e@cyient.com>
<1513161371.5128.3.camel@cybertec.at>
<20171213121952.GA16691@telsasoft.com>
On 12/13/2017 01:19 PM, Justin Pryzby wrote:
> On Wed, Dec 13, 2017 at 11:36:11AM +0100, Laurenz Albe wrote:
>> Dinesh Chandra 12108 wrote:
>>> My CPU utilization is going to 100% in PostgreSQL because of one unknown process /x3303400001 is running from postgres user.
>>>
>>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>>> 19885 postgres 20 0 192684 3916 1420 S 99.3 0.1 5689:04 x3303400001
>>>
>>> The same file is automatically created in Postgres Cluster also. I am using Postgresql-9.3.
>>>
>>> Kindly suggest how can I resolve this issue.
>>
>> I don't know, but the same problem has been reported on Stackoverflow:
>> https://stackoverflow.com/q/46617329/6464308
>>
>> If your queries look similar, then you might indeed be the victim of an attack.
>>
>> Figure out where the function and the executable come from.
>>
>> In case of doubt, disconnect the server from the network.
>
> Looks suspicious; I would look at (and save) things like these:
>
> ls -l /proc/19885/exe
> ls -l /proc/19885/fd
> ls -l /proc/19885/cwd
>
> sudo lsof -n -p 19885
> sudo netstat -anpe |grep 19885
>
> Stacktrace with gcore/gdb is a good idea.
> Save a copy of your log/postgres logfiles and try to figure out where it came
> from. Since an attacker seems to control the postgres process, your data may
> have been compromized (leaked or tampered with).
>
Any details about the x3303400001 file (is it a shell script or some
kind of binary)?
FWIW the queries (listed in the stackoverflow post) are running under
postgres, which I assume is superuser. The backend has full access to
the data directory, of course, so it may create extra files (using
adminpack extension, for example).
If that's the case (and if it's indeed an attack), it either means the
attacker likely already has access to all the data. So presumably
x3303400001 is doing something else at the OS level.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: tomas.vondra@2ndquadrant.com, pryzby@telsasoft.com, laurenz.albe@cybertec.at, Dinesh.Chandra@cyient.com
Subject: Re: CPU 100% usage caused by unknown postgres process..
In-Reply-To: <96e72d15-7edf-96bd-69e1-59ef562f4ca8@2ndquadrant.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox