Re: [Xen-devel] [PATCH v6] tools/xenconsoled: Increase file descriptor limit

[Frediano Ziglio <freddy77@xxxxxxxxx>]

2015-03-02 15:04 GMT+00:00 Andrew Cooper <andrew.cooper3@xxxxxxxxxx>:
> XenServer's VM density testing uncovered a regression when moving from
> sysvinit to systemd where the file descriptor limit dropped from 4096 to
> 1024. (XenServer had previously inserted a ulimit statement into its
> initscripts.)
>
> One solution is to use LimitNOFILE=4096 in xenconsoled.service to match the
> lost ulimit, but that is only a stopgap solution.
>
> As Xenconsoled genuinely needs a large number of file descriptors if a large
> number of domains are running, attempt to increase the limit.
>
> Signed-off-by: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
> Acked-by: Ian Jackson <Ian.Jackson@xxxxxxxxxxxxx>
>

Nothing against the patch but I think you should then check maximum
number of pty on the system (ie
http://www.cyberciti.biz/tips/howto-linux-increase-pty-session.html).
Usually is high (like 4096) but still less than theory Xen domain
limit (about 32k). This (in theory) would lead to domain0 with a not
working interactive ssh.
I would also be curious why the server always keeps slave descriptor open.

Frediano

> ---
> v6:
>  * Fix 32bit build.  (rlimit_t is an arch-specific width)
> v5:
>  * Drop system maximum checking
>  * Unify set paths
> v4:
>  * Calculate fd limit based on domid ABI - result is 132008 fds
>  * Warn if sufficient fds are not available.
> v3:
>  * Hide Linux specific bits in #ifdef __linux__
> v2:
>  * Always increase soft limit to hard limit
>  * Correct commment regarding number of file descriptors
>  * long -> unsigned long as that appears to be the underlying type of an 
> rlim_t
> ---
>  tools/console/daemon/main.c |   38 ++++++++++++++++++++++++++++++++++++++
>  1 file changed, 38 insertions(+)
>
> diff --git a/tools/console/daemon/main.c b/tools/console/daemon/main.c
> index 92d2fc4..11de5c9 100644
> --- a/tools/console/daemon/main.c
> +++ b/tools/console/daemon/main.c
> @@ -26,6 +26,7 @@
>  #include <string.h>
>  #include <signal.h>
>  #include <sys/types.h>
> +#include <sys/resource.h>
>
>  #include "xenctrl.h"
>
> @@ -55,6 +56,41 @@ static void version(char *name)
>         printf("Xen Console Daemon 3.0\n");
>  }
>
> +static void increase_fd_limit(void)
> +{
> +       /*
> +        * We require many file descriptors:
> +        * - per domain: pty master, pty slave, logfile and evtchn
> +        * - misc extra: hypervisor log, privcmd, gntdev, std...
> +        *
> +        * Allow a generous 1000 for misc, and calculate the maximum possible
> +        * number of fds which could be used.
> +        */
> +       unsigned min_fds = (DOMID_FIRST_RESERVED * 4) + 1000;
> +       struct rlimit lim, new = { min_fds, min_fds };
> +
> +       if (getrlimit(RLIMIT_NOFILE, &lim) < 0) {
> +               fprintf(stderr, "Failed to obtain fd limit: %s\n",
> +                       strerror(errno));
> +               exit(1);
> +       }
> +
> +       /* Do we already have sufficient? Great! */
> +       if (lim.rlim_cur >= min_fds)
> +               return;
> +
> +       /* Try to increase our limit. */
> +       if (setrlimit(RLIMIT_NOFILE, &new) < 0)
> +               syslog(LOG_WARNING,
> +                      "Unable to increase fd limit from {%llu, %llu} to "
> +                      "{%llu, %llu}: (%s) - May run out with lots of 
> domains",
> +                      (unsigned long long)lim.rlim_cur,
> +                      (unsigned long long)lim.rlim_max,
> +                      (unsigned long long)new.rlim_cur,
> +                      (unsigned long long)new.rlim_max,
> +                      strerror(errno));
> +}
> +
>  int main(int argc, char **argv)
>  {
>         const char *sopts = "hVvit:o:";
> @@ -154,6 +190,8 @@ int main(int argc, char **argv)
>         openlog("xenconsoled", syslog_option, LOG_DAEMON);
>         setlogmask(syslog_mask);
>
> +       increase_fd_limit();
> +
>         if (!is_interactive) {
>                 daemonize(pidfile ? pidfile : "/var/run/xenconsoled.pid");
>         }
> --
> 1.7.10.4
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> http://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel