LCOV - code coverage report
Current view: top level - gcc - ipa-sra.c (source / functions) Hit Total Coverage
Test: gcc.info Lines: 1735 1898 91.4 %
Date: 2020-04-04 11:58:09 Functions: 74 81 91.4 %
Legend: Lines: hit not hit | Branches: + taken - not taken # not executed Branches: 0 0 -

           Branch data     Line data    Source code
       1                 :            : /* Interprocedural scalar replacement of aggregates
       2                 :            :    Copyright (C) 2008-2020 Free Software Foundation, Inc.
       3                 :            : 
       4                 :            :    Contributed by Martin Jambor <mjambor@suse.cz>
       5                 :            : 
       6                 :            : This file is part of GCC.
       7                 :            : 
       8                 :            : GCC is free software; you can redistribute it and/or modify it under
       9                 :            : the terms of the GNU General Public License as published by the Free
      10                 :            : Software Foundation; either version 3, or (at your option) any later
      11                 :            : version.
      12                 :            : 
      13                 :            : GCC is distributed in the hope that it will be useful, but WITHOUT ANY
      14                 :            : WARRANTY; without even the implied warranty of MERCHANTABILITY or
      15                 :            : FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
      16                 :            : for more details.
      17                 :            : 
      18                 :            : You should have received a copy of the GNU General Public License
      19                 :            : along with GCC; see the file COPYING3.  If not see
      20                 :            : <http://www.gnu.org/licenses/>.  */
      21                 :            : 
      22                 :            : /* IPA-SRA is an interprocedural pass that removes unused function return
      23                 :            :    values (turning functions returning a value which is never used into void
      24                 :            :    functions), removes unused function parameters.  It can also replace an
      25                 :            :    aggregate parameter by a set of other parameters representing part of the
      26                 :            :    original, turning those passed by reference into new ones which pass the
      27                 :            :    value directly.
      28                 :            : 
      29                 :            :    The pass is a true IPA one, which means that it works in three stages in
      30                 :            :    order to be able to take advantage of LTO.  First, summaries about functions
      31                 :            :    and each calls are generated.  Function summaries (often called call graph
      32                 :            :    node summaries) contain mainly information about which parameters are
      33                 :            :    potential transformation candidates and which bits of candidates are
      34                 :            :    accessed.  We differentiate between accesses done as a part of a call
      35                 :            :    statement (which might be not necessary if the callee is also transformed)
      36                 :            :    and others (which are mandatory).  Call summaries (often called call graph
      37                 :            :    edge summaries) contain information about which function formal parameters
      38                 :            :    feed into which actual call arguments so that if two parameters are only
      39                 :            :    used in a sum which is then passed to another function which then however
      40                 :            :    does not use this parameter, all three parameters of the two functions can
      41                 :            :    be eliminated.  Edge summaries also have flags whether the return value is
      42                 :            :    used or if it is only returned in the caller too.  In LTO mode these
      43                 :            :    summaries are then streamed to the object file in the compilation phase and
      44                 :            :    streamed back in in the WPA analysis stage.
      45                 :            : 
      46                 :            :    The interprocedural analysis phase traverses the graph in topological order
      47                 :            :    in two sweeps, one in each direction.  First, from callees to callers for
      48                 :            :    parameter removal and splitting.  Each strongly-connected component is
      49                 :            :    processed iteratively until the situation in it stabilizes.  The pass from
      50                 :            :    callers to callees is then carried out to remove unused return values in a
      51                 :            :    very similar fashion.
      52                 :            : 
      53                 :            :    Because parameter manipulation has big implications for call redirection
      54                 :            :    which is done only after all call graph nodes materialize, the
      55                 :            :    transformation phase is not part of this patch but is carried out by the
      56                 :            :    clone materialization and edge redirection itself, see comments in
      57                 :            :    ipa-param-manipulation.h for more details.  */
      58                 :            : 
      59                 :            : 
      60                 :            : 
      61                 :            : #include "config.h"
      62                 :            : #include "system.h"
      63                 :            : #include "coretypes.h"
      64                 :            : #include "backend.h"
      65                 :            : #include "tree.h"
      66                 :            : #include "gimple.h"
      67                 :            : #include "predict.h"
      68                 :            : #include "tree-pass.h"
      69                 :            : #include "ssa.h"
      70                 :            : #include "cgraph.h"
      71                 :            : #include "gimple-pretty-print.h"
      72                 :            : #include "alias.h"
      73                 :            : #include "tree-eh.h"
      74                 :            : #include "gimple-iterator.h"
      75                 :            : #include "gimple-walk.h"
      76                 :            : #include "tree-dfa.h"
      77                 :            : #include "tree-sra.h"
      78                 :            : #include "alloc-pool.h"
      79                 :            : #include "symbol-summary.h"
      80                 :            : #include "dbgcnt.h"
      81                 :            : #include "tree-inline.h"
      82                 :            : #include "ipa-utils.h"
      83                 :            : #include "builtins.h"
      84                 :            : #include "cfganal.h"
      85                 :            : #include "tree-streamer.h"
      86                 :            : 
      87                 :            : 
      88                 :            : /* Bits used to track size of an aggregate in bytes interprocedurally.  */
      89                 :            : #define ISRA_ARG_SIZE_LIMIT_BITS 16
      90                 :            : #define ISRA_ARG_SIZE_LIMIT (1 << ISRA_ARG_SIZE_LIMIT_BITS)
      91                 :            : /* How many parameters can feed into a call actual argument and still be
      92                 :            :    tracked. */
      93                 :            : #define IPA_SRA_MAX_PARAM_FLOW_LEN 7
      94                 :            : 
      95                 :            : /* Structure describing accesses to a specific portion of an aggregate
      96                 :            :    parameter, as given by the offset and size.  Any smaller accesses that occur
      97                 :            :    within a function that fall within another access form a tree.  The pass
      98                 :            :    cannot analyze parameters with only partially overlapping accesses.  */
      99                 :            : 
     100                 :            : struct GTY(()) param_access
     101                 :            : {
     102                 :            :   /* Type that a potential replacement should have.  This field only has
     103                 :            :      meaning in the summary building and transformation phases, when it is
     104                 :            :      reconstructed from the body.  Must not be touched in IPA analysis
     105                 :            :      stage.  */
     106                 :            :   tree type;
     107                 :            : 
     108                 :            :   /* Alias reference type to be used in MEM_REFs when adjusting caller
     109                 :            :      arguments.  */
     110                 :            :   tree alias_ptr_type;
     111                 :            : 
     112                 :            :   /* Values returned by get_ref_base_and_extent but converted to bytes and
     113                 :            :      stored as unsigned ints.  */
     114                 :            :   unsigned unit_offset;
     115                 :            :   unsigned unit_size : ISRA_ARG_SIZE_LIMIT_BITS;
     116                 :            : 
     117                 :            :   /* Set once we are sure that the access will really end up in a potentially
     118                 :            :      transformed function - initially not set for portions of formal parameters
     119                 :            :      that are only used as actual function arguments passed to callees.  */
     120                 :            :   unsigned certain : 1;
     121                 :            :   /* Set if the access has a reversed scalar storage order.  */
     122                 :            :   unsigned reverse : 1;
     123                 :            : };
     124                 :            : 
     125                 :            : /* This structure has the same purpose as the one above and additionally it
     126                 :            :    contains some fields that are only necessary in the summary generation
     127                 :            :    phase.  */
     128                 :            : 
     129                 :            : struct gensum_param_access
     130                 :            : {
     131                 :            :   /* Values returned by get_ref_base_and_extent.  */
     132                 :            :   HOST_WIDE_INT offset;
     133                 :            :   HOST_WIDE_INT size;
     134                 :            : 
     135                 :            :   /* if this access has any children (in terms of the definition above), this
     136                 :            :      points to the first one.  */
     137                 :            :   struct gensum_param_access *first_child;
     138                 :            :   /* In intraprocedural SRA, pointer to the next sibling in the access tree as
     139                 :            :      described above.  */
     140                 :            :   struct gensum_param_access *next_sibling;
     141                 :            : 
     142                 :            :   /* Type that a potential replacement should have.  This field only has
     143                 :            :      meaning in the summary building and transformation phases, when it is
     144                 :            :      reconstructed from the body.  Must not be touched in IPA analysis
     145                 :            :      stage.  */
     146                 :            :   tree type;
     147                 :            :   /* Alias reference type to be used in MEM_REFs when adjusting caller
     148                 :            :      arguments.  */
     149                 :            :   tree alias_ptr_type;
     150                 :            : 
     151                 :            :   /* Have there been writes to or reads from this exact location except for as
     152                 :            :      arguments to a function call that can be tracked.  */
     153                 :            :   bool nonarg;
     154                 :            : 
     155                 :            :   /* Set if the access has a reversed scalar storage order.  */
     156                 :            :   bool reverse;
     157                 :            : };
     158                 :            : 
     159                 :            : /* Summary describing a parameter in the IPA stages.  */
     160                 :            : 
     161                 :            : struct GTY(()) isra_param_desc
     162                 :            : {
     163                 :            :   /* List of access representatives to the parameters, sorted according to
     164                 :            :      their offset.  */
     165                 :            :   vec <param_access *, va_gc> *accesses;
     166                 :            : 
     167                 :            :   /* Unit size limit of total size of all replacements.  */
     168                 :            :   unsigned param_size_limit : ISRA_ARG_SIZE_LIMIT_BITS;
     169                 :            :   /* Sum of unit sizes of all certain replacements.  */
     170                 :            :   unsigned size_reached : ISRA_ARG_SIZE_LIMIT_BITS;
     171                 :            : 
     172                 :            :   /* A parameter that is used only in call arguments and can be removed if all
     173                 :            :      concerned actual arguments are removed.  */
     174                 :            :   unsigned locally_unused : 1;
     175                 :            :   /* An aggregate that is a candidate for breaking up or complete removal.  */
     176                 :            :   unsigned split_candidate : 1;
     177                 :            :   /* Is this a parameter passing stuff by reference?  */
     178                 :            :   unsigned by_ref : 1;
     179                 :            : };
     180                 :            : 
     181                 :            : /* Structure used when generating summaries that describes a parameter.  */
     182                 :            : 
     183                 :            : struct gensum_param_desc
     184                 :            : {
     185                 :            :   /* Roots of param_accesses.  */
     186                 :            :   gensum_param_access *accesses;
     187                 :            :   /* Number of accesses in the access tree rooted in field accesses.  */
     188                 :            :   unsigned access_count;
     189                 :            : 
     190                 :            :   /* If the below is non-zero, this is the number of uses as actual arguments.  */
     191                 :            :   int call_uses;
     192                 :            :   /* Number of times this parameter has been directly passed to.  */
     193                 :            :   unsigned ptr_pt_count;
     194                 :            : 
     195                 :            :   /* Size limit of total size of all replacements.  */
     196                 :            :   unsigned param_size_limit;
     197                 :            :   /* Sum of sizes of nonarg accesses.  */
     198                 :            :   unsigned nonarg_acc_size;
     199                 :            : 
     200                 :            :   /* A parameter that is used only in call arguments and can be removed if all
     201                 :            :      concerned actual arguments are removed.  */
     202                 :            :   bool locally_unused;
     203                 :            :   /* An aggregate that is a candidate for breaking up or a pointer passing data
     204                 :            :      by reference that is a candidate for being converted to a set of
     205                 :            :      parameters passing those data by value.  */
     206                 :            :   bool split_candidate;
     207                 :            :   /* Is this a parameter passing stuff by reference?  */
     208                 :            :   bool by_ref;
     209                 :            : 
     210                 :            :   /* The number of this parameter as they are ordered in function decl.  */
     211                 :            :   int param_number;
     212                 :            :   /* For parameters passing data by reference, this is parameter index to
     213                 :            :      compute indices to bb_dereferences.  */
     214                 :            :   int deref_index;
     215                 :            : };
     216                 :            : 
     217                 :            : /* Properly deallocate accesses of DESC.  TODO: Since this data structure is
     218                 :            :    not in GC memory, this is not necessary and we can consider removing the
     219                 :            :    function.  */
     220                 :            : 
     221                 :            : static void
     222                 :    1611910 : free_param_decl_accesses (isra_param_desc *desc)
     223                 :            : {
     224                 :    1611910 :   unsigned len = vec_safe_length (desc->accesses);
     225                 :    1923410 :   for (unsigned i = 0; i < len; ++i)
     226                 :     311498 :     ggc_free ((*desc->accesses)[i]);
     227                 :    1611910 :   vec_free (desc->accesses);
     228                 :    1611910 : }
     229                 :            : 
     230                 :            : /* Class used to convey information about functions from the
     231                 :            :    intra-procedural analysis stage to inter-procedural one.  */
     232                 :            : 
     233                 :            : class GTY((for_user)) isra_func_summary
     234                 :            : {
     235                 :            : public:
     236                 :            :   /* initialize the object.  */
     237                 :            : 
     238                 :     720107 :   isra_func_summary ()
     239                 :     720107 :     : m_parameters (NULL), m_candidate (false), m_returns_value (false),
     240                 :     720107 :     m_return_ignored (false), m_queued (false)
     241                 :            :   {}
     242                 :            : 
     243                 :            :   /* Destroy m_parameters.  */
     244                 :            : 
     245                 :            :   ~isra_func_summary ();
     246                 :            : 
     247                 :            :   /* Mark the function as not a candidate for any IPA-SRA transformation.
     248                 :            :      Return true if it was a candidate until now.  */
     249                 :            : 
     250                 :            :   bool zap ();
     251                 :            : 
     252                 :            :   /* Vector of parameter descriptors corresponding to the function being
     253                 :            :      analyzed.  */
     254                 :            :   vec<isra_param_desc, va_gc> *m_parameters;
     255                 :            : 
     256                 :            :   /* Whether the node is even a candidate for any IPA-SRA transformation at
     257                 :            :      all.  */
     258                 :            :   unsigned m_candidate : 1;
     259                 :            : 
     260                 :            :   /* Whether the original function returns any value.  */
     261                 :            :   unsigned m_returns_value : 1;
     262                 :            : 
     263                 :            :   /* Set to true if all call statements do not actually use the returned
     264                 :            :      value.  */
     265                 :            : 
     266                 :            :   unsigned m_return_ignored : 1;
     267                 :            : 
     268                 :            :   /* Whether the node is already queued in IPA SRA stack during processing of
     269                 :            :      call graphs SCCs.  */
     270                 :            : 
     271                 :            :   unsigned m_queued : 1;
     272                 :            : };
     273                 :            : 
     274                 :            : /* Clean up and deallocate isra_func_summary points to.  TODO: Since this data
     275                 :            :    structure is not in GC memory, this is not necessary and we can consider
     276                 :            :    removing the destructor.  */
     277                 :            : 
     278                 :    1407920 : isra_func_summary::~isra_func_summary ()
     279                 :            : {
     280                 :     703960 :   unsigned len = vec_safe_length (m_parameters);
     281                 :    1323380 :   for (unsigned i = 0; i < len; ++i)
     282                 :     619421 :     free_param_decl_accesses (&(*m_parameters)[i]);
     283                 :     703960 :   vec_free (m_parameters);
     284                 :     703960 : }
     285                 :            : 
     286                 :            : 
     287                 :            : /* Mark the function as not a candidate for any IPA-SRA transformation.  Return
     288                 :            :    true if it was a candidate until now.  */
     289                 :            : 
     290                 :            : bool
     291                 :     397061 : isra_func_summary::zap ()
     292                 :            : {
     293                 :     397061 :   bool ret = m_candidate;
     294                 :     397061 :   m_candidate = false;
     295                 :            : 
     296                 :     397061 :   unsigned len = vec_safe_length (m_parameters);
     297                 :    1389550 :   for (unsigned i = 0; i < len; ++i)
     298                 :     992488 :     free_param_decl_accesses (&(*m_parameters)[i]);
     299                 :     397061 :   vec_free (m_parameters);
     300                 :            : 
     301                 :     397061 :   return ret;
     302                 :            : }
     303                 :            : 
     304                 :            : /* Structure to describe which formal parameters feed into a particular actual
     305                 :            :    arguments.  */
     306                 :            : 
     307                 :            : struct isra_param_flow
     308                 :            : {
     309                 :            :   /* Number of elements in array inputs that contain valid data.  */
     310                 :            :   char length;
     311                 :            :   /* Indices of formal parameters that feed into the described actual argument.
     312                 :            :      If aggregate_pass_through or pointer_pass_through below are true, it must
     313                 :            :      contain exactly one element which is passed through from a formal
     314                 :            :      parameter if the given number.  Otherwise, the array contains indices of
     315                 :            :      callee's formal parameters which are used to calculate value of this
     316                 :            :      actual argument. */
     317                 :            :   unsigned char inputs[IPA_SRA_MAX_PARAM_FLOW_LEN];
     318                 :            : 
     319                 :            :   /* Offset within the formal parameter.  */
     320                 :            :   unsigned unit_offset;
     321                 :            :   /* Size of the portion of the formal parameter that is being passed.  */
     322                 :            :   unsigned unit_size : ISRA_ARG_SIZE_LIMIT_BITS;
     323                 :            : 
     324                 :            :   /* True when the value of this actual copy is a portion of a formal
     325                 :            :      parameter.  */
     326                 :            :   unsigned aggregate_pass_through : 1;
     327                 :            :   /* True when the value of this actual copy is a verbatim pass through of an
     328                 :            :      obtained pointer.  */
     329                 :            :   unsigned pointer_pass_through : 1;
     330                 :            :   /* True when it is safe to copy access candidates here from the callee, which
     331                 :            :      would mean introducing dereferences into callers of the caller.  */
     332                 :            :   unsigned safe_to_import_accesses : 1;
     333                 :            : };
     334                 :            : 
     335                 :            : /* Structure used to convey information about calls from the intra-procedural
     336                 :            :    analysis stage to inter-procedural one.  */
     337                 :            : 
     338                 :    3884040 : class isra_call_summary
     339                 :            : {
     340                 :            : public:
     341                 :    3081460 :   isra_call_summary ()
     342                 :    3081460 :     : m_arg_flow (), m_return_ignored (false), m_return_returned (false),
     343                 :    3081460 :       m_bit_aligned_arg (false)
     344                 :            :   {}
     345                 :            : 
     346                 :            :   void init_inputs (unsigned arg_count);
     347                 :            :   void dump (FILE *f);
     348                 :            : 
     349                 :            :   /* Information about what formal parameters of the caller are used to compute
     350                 :            :      individual actual arguments of this call.  */
     351                 :            :   auto_vec <isra_param_flow> m_arg_flow;
     352                 :            : 
     353                 :            :   /* Set to true if the call statement does not have a LHS.  */
     354                 :            :   unsigned m_return_ignored : 1;
     355                 :            : 
     356                 :            :   /* Set to true if the LHS of call statement is only used to construct the
     357                 :            :      return value of the caller.  */
     358                 :            :   unsigned m_return_returned : 1;
     359                 :            : 
     360                 :            :   /* Set when any of the call arguments are not byte-aligned.  */
     361                 :            :   unsigned m_bit_aligned_arg : 1;
     362                 :            : };
     363                 :            : 
     364                 :            : /* Class to manage function summaries.  */
     365                 :            : 
     366                 :            : class GTY((user)) ipa_sra_function_summaries
     367                 :            :   : public function_summary <isra_func_summary *>
     368                 :            : {
     369                 :            : public:
     370                 :     100514 :   ipa_sra_function_summaries (symbol_table *table, bool ggc):
     371                 :     100514 :     function_summary<isra_func_summary *> (table, ggc) { }
     372                 :            : 
     373                 :            :   virtual void duplicate (cgraph_node *, cgraph_node *,
     374                 :            :                           isra_func_summary *old_sum,
     375                 :            :                           isra_func_summary *new_sum);
     376                 :            : };
     377                 :            : 
     378                 :            : /* Hook that is called by summary when a node is duplicated.  */
     379                 :            : 
     380                 :            : void
     381                 :      69251 : ipa_sra_function_summaries::duplicate (cgraph_node *, cgraph_node *,
     382                 :            :                                        isra_func_summary *old_sum,
     383                 :            :                                        isra_func_summary *new_sum)
     384                 :            : {
     385                 :            :   /* TODO: Somehow stop copying when ISRA is doing the cloning, it is
     386                 :            :      useless.  */
     387                 :      69251 :   new_sum->m_candidate  = old_sum->m_candidate;
     388                 :      69251 :   new_sum->m_returns_value = old_sum->m_returns_value;
     389                 :      69251 :   new_sum->m_return_ignored = old_sum->m_return_ignored;
     390                 :      69251 :   gcc_assert (!old_sum->m_queued);
     391                 :      69251 :   new_sum->m_queued = false;
     392                 :            : 
     393                 :      69251 :   unsigned param_count = vec_safe_length (old_sum->m_parameters);
     394                 :      68823 :   if (!param_count)
     395                 :            :     return;
     396                 :      68823 :   vec_safe_reserve_exact (new_sum->m_parameters, param_count);
     397                 :      68823 :   new_sum->m_parameters->quick_grow_cleared (param_count);
     398                 :     259784 :   for (unsigned i = 0; i < param_count; i++)
     399                 :            :     {
     400                 :     190961 :       isra_param_desc *s = &(*old_sum->m_parameters)[i];
     401                 :     190961 :       isra_param_desc *d = &(*new_sum->m_parameters)[i];
     402                 :            : 
     403                 :     190961 :       d->param_size_limit = s->param_size_limit;
     404                 :     190961 :       d->size_reached = s->size_reached;
     405                 :     190961 :       d->locally_unused = s->locally_unused;
     406                 :     190961 :       d->split_candidate = s->split_candidate;
     407                 :     190961 :       d->by_ref = s->by_ref;
     408                 :            : 
     409                 :     190961 :       unsigned acc_count = vec_safe_length (s->accesses);
     410                 :     190961 :       vec_safe_reserve_exact (d->accesses, acc_count);
     411                 :     245315 :       for (unsigned j = 0; j < acc_count; j++)
     412                 :            :         {
     413                 :      54354 :           param_access *from = (*s->accesses)[j];
     414                 :      54354 :           param_access *to = ggc_cleared_alloc<param_access> ();
     415                 :      54354 :           to->type = from->type;
     416                 :      54354 :           to->alias_ptr_type = from->alias_ptr_type;
     417                 :      54354 :           to->unit_offset = from->unit_offset;
     418                 :      54354 :           to->unit_size = from->unit_size;
     419                 :      54354 :           to->certain = from->certain;
     420                 :      54354 :           d->accesses->quick_push (to);
     421                 :            :         }
     422                 :            :     }
     423                 :            : }
     424                 :            : 
     425                 :            : /* Pointer to the pass function summary holder.  */
     426                 :            : 
     427                 :            : static GTY(()) ipa_sra_function_summaries *func_sums;
     428                 :            : 
     429                 :            : /* Class to manage call summaries.  */
     430                 :            : 
     431                 :            : class ipa_sra_call_summaries: public call_summary <isra_call_summary *>
     432                 :            : {
     433                 :            : public:
     434                 :     100514 :   ipa_sra_call_summaries (symbol_table *table):
     435                 :     201028 :     call_summary<isra_call_summary *> (table) { }
     436                 :            : 
     437                 :            :   /* Duplicate info when an edge is cloned.  */
     438                 :            :   virtual void duplicate (cgraph_edge *, cgraph_edge *,
     439                 :            :                           isra_call_summary *old_sum,
     440                 :            :                           isra_call_summary *new_sum);
     441                 :            : };
     442                 :            : 
     443                 :            : static ipa_sra_call_summaries *call_sums;
     444                 :            : 
     445                 :            : 
     446                 :            : /* Initialize m_arg_flow of a particular instance of isra_call_summary.
     447                 :            :    ARG_COUNT is the number of actual arguments passed.  */
     448                 :            : 
     449                 :            : void
     450                 :    1854980 : isra_call_summary::init_inputs (unsigned arg_count)
     451                 :            : {
     452                 :    1854980 :   if (arg_count == 0)
     453                 :            :     {
     454                 :     253401 :       gcc_checking_assert (m_arg_flow.length () == 0);
     455                 :            :       return;
     456                 :            :     }
     457                 :    1601580 :   if (m_arg_flow.length () == 0)
     458                 :            :     {
     459                 :     868633 :       m_arg_flow.reserve_exact (arg_count);
     460                 :     868633 :       m_arg_flow.quick_grow_cleared (arg_count);
     461                 :            :     }
     462                 :            :   else
     463                 :     732948 :     gcc_checking_assert (arg_count == m_arg_flow.length ());
     464                 :            : }
     465                 :            : 
     466                 :            : /* Dump all information in call summary to F.  */
     467                 :            : 
     468                 :            : void
     469                 :         89 : isra_call_summary::dump (FILE *f)
     470                 :            : {
     471                 :         89 :   if (m_return_ignored)
     472                 :         56 :     fprintf (f, "    return value ignored\n");
     473                 :         89 :   if (m_return_returned)
     474                 :         30 :     fprintf (f, "    return value used only to compute caller return value\n");
     475                 :        355 :   for (unsigned i = 0; i < m_arg_flow.length (); i++)
     476                 :            :     {
     477                 :        114 :       fprintf (f, "    Parameter %u:\n", i);
     478                 :        114 :       isra_param_flow *ipf = &m_arg_flow[i];
     479                 :            : 
     480                 :        114 :       if (ipf->length)
     481                 :            :         {
     482                 :        109 :           bool first = true;
     483                 :        109 :           fprintf (f, "      Scalar param sources: ");
     484                 :        220 :           for (int j = 0; j < ipf->length; j++)
     485                 :            :             {
     486                 :        111 :               if (!first)
     487                 :          2 :                 fprintf (f, ", ");
     488                 :            :               else
     489                 :            :                 first = false;
     490                 :        111 :               fprintf (f, "%i", (int) ipf->inputs[j]);
     491                 :            :             }
     492                 :        109 :           fprintf (f, "\n");
     493                 :            :         }
     494                 :        114 :       if (ipf->aggregate_pass_through)
     495                 :         12 :         fprintf (f, "      Aggregate pass through from the param given above, "
     496                 :            :                  "unit offset: %u , unit size: %u\n",
     497                 :         12 :                  ipf->unit_offset, ipf->unit_size);
     498                 :        114 :       if (ipf->pointer_pass_through)
     499                 :          3 :         fprintf (f, "      Pointer pass through from the param given above, "
     500                 :          3 :                  "safe_to_import_accesses: %u\n", ipf->safe_to_import_accesses);
     501                 :            :     }
     502                 :         89 : }
     503                 :            : 
     504                 :            : /* Duplicate edge summary when an edge is cloned.  */
     505                 :            : 
     506                 :            : void
     507                 :     195964 : ipa_sra_call_summaries::duplicate (cgraph_edge *, cgraph_edge *,
     508                 :            :                                    isra_call_summary *old_sum,
     509                 :            :                                    isra_call_summary *new_sum)
     510                 :            : {
     511                 :     195964 :   unsigned arg_count = old_sum->m_arg_flow.length ();
     512                 :     195964 :   new_sum->init_inputs (arg_count);
     513                 :     384988 :   for (unsigned i = 0; i < arg_count; i++)
     514                 :     189024 :     new_sum->m_arg_flow[i] = old_sum->m_arg_flow[i];
     515                 :            : 
     516                 :     195964 :   new_sum->m_return_ignored = old_sum->m_return_ignored;
     517                 :     195964 :   new_sum->m_return_returned = old_sum->m_return_returned;
     518                 :     195964 :   new_sum->m_bit_aligned_arg = old_sum->m_bit_aligned_arg;
     519                 :     195964 : }
     520                 :            : 
     521                 :            : 
     522                 :            : /* With all GTY stuff done, we can move to anonymous namespace.  */
     523                 :            : namespace {
     524                 :            : /* Quick mapping from a decl to its param descriptor.  */
     525                 :            : 
     526                 :            : hash_map<tree, gensum_param_desc *> *decl2desc;
     527                 :            : 
     528                 :            : /* Countdown of allowed Alias analysis steps during summary building.  */
     529                 :            : 
     530                 :            : int aa_walking_limit;
     531                 :            : 
     532                 :            : /* This is a table in which for each basic block and parameter there is a
     533                 :            :    distance (offset + size) in that parameter which is dereferenced and
     534                 :            :    accessed in that BB.  */
     535                 :            : HOST_WIDE_INT *bb_dereferences = NULL;
     536                 :            : /* How many by-reference parameters there are in the current function.  */
     537                 :            : int by_ref_count;
     538                 :            : 
     539                 :            : /* Bitmap of BBs that can cause the function to "stop" progressing by
     540                 :            :    returning, throwing externally, looping infinitely or calling a function
     541                 :            :    which might abort etc.. */
     542                 :            : bitmap final_bbs;
     543                 :            : 
     544                 :            : /* Obstack to allocate various small structures required only when generating
     545                 :            :    summary for a function.  */
     546                 :            : struct obstack gensum_obstack;
     547                 :            : 
     548                 :            : /* Return false the function is apparently unsuitable for IPA-SRA based on it's
     549                 :            :    attributes, return true otherwise.  NODE is the cgraph node of the current
     550                 :            :    function.  */
     551                 :            : 
     552                 :            : static bool
     553                 :     807382 : ipa_sra_preliminary_function_checks (cgraph_node *node)
     554                 :            : {
     555                 :     807382 :   if (!node->can_change_signature)
     556                 :            :     {
     557                 :      50529 :       if (dump_file)
     558                 :          0 :         fprintf (dump_file, "Function cannot change signature.\n");
     559                 :      50529 :       return false;
     560                 :            :     }
     561                 :            : 
     562                 :     756853 :   if (!tree_versionable_function_p (node->decl))
     563                 :            :     {
     564                 :      90079 :       if (dump_file)
     565                 :          5 :         fprintf (dump_file, "Function is not versionable.\n");
     566                 :      90079 :       return false;
     567                 :            :     }
     568                 :            : 
     569                 :     666774 :   if (!opt_for_fn (node->decl, optimize)
     570                 :     666774 :       || !opt_for_fn (node->decl, flag_ipa_sra))
     571                 :            :     {
     572                 :         80 :       if (dump_file)
     573                 :          0 :         fprintf (dump_file, "Not optimizing or IPA-SRA turned off for this "
     574                 :            :                  "function.\n");
     575                 :         80 :       return false;
     576                 :            :     }
     577                 :            : 
     578                 :     666694 :   if (DECL_VIRTUAL_P (node->decl))
     579                 :            :     {
     580                 :      24743 :       if (dump_file)
     581                 :          0 :         fprintf (dump_file, "Function is a virtual method.\n");
     582                 :      24743 :       return false;
     583                 :            :     }
     584                 :            : 
     585                 :     641951 :   struct function *fun = DECL_STRUCT_FUNCTION (node->decl);
     586                 :     641951 :   if (fun->stdarg)
     587                 :            :     {
     588                 :         83 :       if (dump_file)
     589                 :          0 :         fprintf (dump_file, "Function uses stdarg. \n");
     590                 :         83 :       return false;
     591                 :            :     }
     592                 :            : 
     593                 :     641868 :   if (TYPE_ATTRIBUTES (TREE_TYPE (node->decl)))
     594                 :            :     {
     595                 :          0 :       if (dump_file)
     596                 :          0 :         fprintf (dump_file, "Function type has attributes. \n");
     597                 :          0 :       return false;
     598                 :            :     }
     599                 :            : 
     600                 :     641868 :   if (DECL_DISREGARD_INLINE_LIMITS (node->decl))
     601                 :            :     {
     602                 :      33741 :       if (dump_file)
     603                 :          0 :         fprintf (dump_file, "Always inline function will be inlined "
     604                 :            :                  "anyway. \n");
     605                 :      33741 :       return false;
     606                 :            :     }
     607                 :            : 
     608                 :            :   return true;
     609                 :            : }
     610                 :            : 
     611                 :            : /* Print access tree starting at ACCESS to F.  */
     612                 :            : 
     613                 :            : static void
     614                 :         41 : dump_gensum_access (FILE *f, gensum_param_access *access, unsigned indent)
     615                 :            : {
     616                 :         41 :   fprintf (f, "  ");
     617                 :        127 :   for (unsigned i = 0; i < indent; i++)
     618                 :         86 :     fprintf (f, " ");
     619                 :         41 :   fprintf (f, "    * Access to offset: " HOST_WIDE_INT_PRINT_DEC,
     620                 :            :            access->offset);
     621                 :         41 :   fprintf (f, ", size: " HOST_WIDE_INT_PRINT_DEC, access->size);
     622                 :         41 :   fprintf (f, ", type: ");
     623                 :         41 :   print_generic_expr (f, access->type);
     624                 :         41 :   fprintf (f, ", alias_ptr_type: ");
     625                 :         41 :   print_generic_expr (f, access->alias_ptr_type);
     626                 :         41 :   fprintf (f, ", nonarg: %u, reverse: %u\n", access->nonarg, access->reverse);
     627                 :         41 :   for (gensum_param_access *ch = access->first_child;
     628                 :         43 :        ch;
     629                 :          2 :        ch = ch->next_sibling)
     630                 :          2 :     dump_gensum_access (f, ch, indent + 2);
     631                 :         41 : }
     632                 :            : 
     633                 :            : 
     634                 :            : /* Print access tree starting at ACCESS to F.  */
     635                 :            : 
     636                 :            : static void
     637                 :         60 : dump_isra_access (FILE *f, param_access *access)
     638                 :            : {
     639                 :         60 :   fprintf (f, "    * Access to unit offset: %u", access->unit_offset);
     640                 :         60 :   fprintf (f, ", unit size: %u", access->unit_size);
     641                 :         60 :   fprintf (f, ", type: ");
     642                 :         60 :   print_generic_expr (f, access->type);
     643                 :         60 :   fprintf (f, ", alias_ptr_type: ");
     644                 :         60 :   print_generic_expr (f, access->alias_ptr_type);
     645                 :         60 :   if (access->certain)
     646                 :         38 :     fprintf (f, ", certain");
     647                 :            :   else
     648                 :         22 :     fprintf (f, ", not-certain");
     649                 :         60 :   if (access->reverse)
     650                 :          0 :     fprintf (f, ", reverse");
     651                 :         60 :   fprintf (f, "\n");
     652                 :         60 : }
     653                 :            : 
     654                 :            : /* Dump access tree starting at ACCESS to stderr.  */
     655                 :            : 
     656                 :            : DEBUG_FUNCTION void
     657                 :          0 : debug_isra_access (param_access *access)
     658                 :            : {
     659                 :          0 :   dump_isra_access (stderr, access);
     660                 :          0 : }
     661                 :            : 
     662                 :            : /* Dump DESC to F.  */
     663                 :            : 
     664                 :            : static void
     665                 :        144 : dump_gensum_param_descriptor (FILE *f, gensum_param_desc *desc)
     666                 :            : {
     667                 :        144 :   if (desc->locally_unused)
     668                 :         38 :     fprintf (f, "    unused with %i call_uses\n", desc->call_uses);
     669                 :        144 :   if (!desc->split_candidate)
     670                 :            :     {
     671                 :         96 :       fprintf (f, "    not a candidate\n");
     672                 :         96 :       return;
     673                 :            :     }
     674                 :         48 :   if (desc->by_ref)
     675                 :         33 :     fprintf (f, "    by_ref with %u pass throughs\n", desc->ptr_pt_count);
     676                 :            : 
     677                 :         87 :   for (gensum_param_access *acc = desc->accesses; acc; acc = acc->next_sibling)
     678                 :         39 :     dump_gensum_access (f, acc, 2);
     679                 :            : }
     680                 :            : 
     681                 :            : /* Dump all parameter descriptors in IFS, assuming it describes FNDECL, to
     682                 :            :    F.  */
     683                 :            : 
     684                 :            : static void
     685                 :         43 : dump_gensum_param_descriptors (FILE *f, tree fndecl,
     686                 :            :                                vec<gensum_param_desc> *param_descriptions)
     687                 :            : {
     688                 :         43 :   tree parm = DECL_ARGUMENTS (fndecl);
     689                 :        187 :   for (unsigned i = 0;
     690                 :        374 :        i < param_descriptions->length ();
     691                 :        144 :        ++i, parm = DECL_CHAIN (parm))
     692                 :            :     {
     693                 :        144 :       fprintf (f, "  Descriptor for parameter %i ", i);
     694                 :        144 :       print_generic_expr (f, parm, TDF_UID);
     695                 :        144 :       fprintf (f, "\n");
     696                 :        144 :       dump_gensum_param_descriptor (f, &(*param_descriptions)[i]);
     697                 :            :     }
     698                 :         43 : }
     699                 :            : 
     700                 :            : 
     701                 :            : /* Dump DESC to F.   */
     702                 :            : 
     703                 :            : static void
     704                 :        431 : dump_isra_param_descriptor (FILE *f, isra_param_desc *desc)
     705                 :            : {
     706                 :        431 :   if (desc->locally_unused)
     707                 :            :     {
     708                 :        210 :       fprintf (f, "    (locally) unused\n");
     709                 :            :     }
     710                 :        431 :   if (!desc->split_candidate)
     711                 :            :     {
     712                 :        358 :       fprintf (f, "    not a candidate for splitting\n");
     713                 :        358 :       return;
     714                 :            :     }
     715                 :        146 :   fprintf (f, "    param_size_limit: %u, size_reached: %u%s\n",
     716                 :         73 :            desc->param_size_limit, desc->size_reached,
     717                 :         73 :            desc->by_ref ? ", by_ref" : "");
     718                 :            : 
     719                 :        238 :   for (unsigned i = 0; i < vec_safe_length (desc->accesses); ++i)
     720                 :            :     {
     721                 :         60 :       param_access *access = (*desc->accesses)[i];
     722                 :         60 :       dump_isra_access (f, access);
     723                 :            :     }
     724                 :            : }
     725                 :            : 
     726                 :            : /* Dump all parameter descriptors in IFS, assuming it describes FNDECL, to
     727                 :            :    F.  */
     728                 :            : 
     729                 :            : static void
     730                 :         78 : dump_isra_param_descriptors (FILE *f, tree fndecl,
     731                 :            :                              isra_func_summary *ifs)
     732                 :            : {
     733                 :         78 :   tree parm = DECL_ARGUMENTS (fndecl);
     734                 :         78 :   if (!ifs->m_parameters)
     735                 :            :     {
     736                 :          0 :       fprintf (f, "  parameter descriptors not available\n");
     737                 :          0 :       return;
     738                 :            :     }
     739                 :            : 
     740                 :        222 :   for (unsigned i = 0;
     741                 :        300 :        i < ifs->m_parameters->length ();
     742                 :        222 :        ++i, parm = DECL_CHAIN (parm))
     743                 :            :     {
     744                 :        222 :       fprintf (f, "  Descriptor for parameter %i ", i);
     745                 :        222 :       print_generic_expr (f, parm, TDF_UID);
     746                 :        222 :       fprintf (f, "\n");
     747                 :        222 :       dump_isra_param_descriptor (f, &(*ifs->m_parameters)[i]);
     748                 :            :     }
     749                 :            : }
     750                 :            : 
     751                 :            : /* Add SRC to inputs of PARAM_FLOW, unless it would exceed storage.  If the
     752                 :            :    function fails return false, otherwise return true.  SRC must fit into an
     753                 :            :    unsigned char.  Used for purposes of transitive unused parameter
     754                 :            :    removal.  */
     755                 :            : 
     756                 :            : static bool
     757                 :     664064 : add_src_to_param_flow (isra_param_flow *param_flow, int src)
     758                 :            : {
     759                 :          0 :   gcc_checking_assert (src >= 0 && src <= UCHAR_MAX);
     760                 :     664064 :   if (param_flow->length == IPA_SRA_MAX_PARAM_FLOW_LEN)
     761                 :            :     return false;
     762                 :            : 
     763                 :     664062 :   param_flow->inputs[(int) param_flow->length] = src;
     764                 :     664062 :   param_flow->length++;
     765                 :          0 :   return true;
     766                 :            : }
     767                 :            : 
     768                 :            : /* Add a SRC to the inputs of PARAM_FLOW unless it is already there and assert
     769                 :            :    it is the only input.  Used for purposes of transitive parameter
     770                 :            :    splitting.  */
     771                 :            : 
     772                 :            : static void
     773                 :     361223 : set_single_param_flow_source (isra_param_flow *param_flow, int src)
     774                 :            : {
     775                 :     361223 :   gcc_checking_assert (src >= 0 && src <= UCHAR_MAX);
     776                 :     361223 :   if (param_flow->length == 0)
     777                 :            :     {
     778                 :      75459 :       param_flow->inputs[0] = src;
     779                 :      75459 :       param_flow->length = 1;
     780                 :            :     }
     781                 :     285764 :   else if (param_flow->length == 1)
     782                 :     285764 :     gcc_assert (param_flow->inputs[0] == src);
     783                 :            :   else
     784                 :          0 :     gcc_unreachable ();
     785                 :     361223 : }
     786                 :            : 
     787                 :            : /* Assert that there is only a single value in PARAM_FLOW's inputs and return
     788                 :            :    it.  */
     789                 :            : 
     790                 :            : static unsigned
     791                 :     331557 : get_single_param_flow_source (const isra_param_flow *param_flow)
     792                 :            : {
     793                 :     178287 :   gcc_assert (param_flow->length == 1);
     794                 :     331557 :   return param_flow->inputs[0];
     795                 :            : }
     796                 :            : 
     797                 :            : /* Inspect all uses of NAME and simple arithmetic calculations involving NAME
     798                 :            :    in NODE and return a negative number if any of them is used for something
     799                 :            :    else than either an actual call argument, simple arithmetic operation or
     800                 :            :    debug statement.  If there are no such uses, return the number of actual
     801                 :            :    arguments that this parameter eventually feeds to (or zero if there is none).
     802                 :            :    For any such parameter, mark PARM_NUM as one of its sources.  ANALYZED is a
     803                 :            :    bitmap that tracks which SSA names we have already started
     804                 :            :    investigating.  */
     805                 :            : 
     806                 :            : static int
     807                 :    1671060 : isra_track_scalar_value_uses (cgraph_node *node, tree name, int parm_num,
     808                 :            :                               bitmap analyzed)
     809                 :            : {
     810                 :    1671060 :   int res = 0;
     811                 :    1671060 :   imm_use_iterator imm_iter;
     812                 :    1671060 :   gimple *stmt;
     813                 :            : 
     814                 :    3107210 :   FOR_EACH_IMM_USE_STMT (stmt, imm_iter, name)
     815                 :            :     {
     816                 :    2601120 :       if (is_gimple_debug (stmt))
     817                 :     500537 :         continue;
     818                 :            : 
     819                 :            :       /* TODO: We could handle at least const builtin functions like arithmetic
     820                 :            :          operations below.  */
     821                 :    2100580 :       if (is_gimple_call (stmt))
     822                 :            :         {
     823                 :     675980 :           int all_uses = 0;
     824                 :     675980 :           use_operand_p use_p;
     825                 :    1362940 :           FOR_EACH_IMM_USE_ON_STMT (use_p, imm_iter)
     826                 :     686957 :             all_uses++;
     827                 :            : 
     828                 :     675980 :           gcall *call = as_a <gcall *> (stmt);
     829                 :     675980 :           unsigned arg_count;
     830                 :     675980 :           if (gimple_call_internal_p (call)
     831                 :     675980 :               || (arg_count = gimple_call_num_args (call)) == 0)
     832                 :            :             {
     833                 :       5941 :               res = -1;
     834                 :       5941 :               BREAK_FROM_IMM_USE_STMT (imm_iter);
     835                 :            :             }
     836                 :            : 
     837                 :     670039 :           cgraph_edge *cs = node->get_edge (stmt);
     838                 :     670039 :           gcc_checking_assert (cs);
     839                 :     670039 :           isra_call_summary *csum = call_sums->get_create (cs);
     840                 :     670039 :           csum->init_inputs (arg_count);
     841                 :            : 
     842                 :     670039 :           int simple_uses = 0;
     843                 :    2864650 :           for (unsigned i = 0; i < arg_count; i++)
     844                 :    2194610 :             if (gimple_call_arg (call, i) == name)
     845                 :            :               {
     846                 :     664064 :                 if (!add_src_to_param_flow (&csum->m_arg_flow[i], parm_num))
     847                 :            :                   {
     848                 :            :                     simple_uses = -1;
     849                 :            :                     break;
     850                 :            :                   }
     851                 :     664062 :                 simple_uses++;
     852                 :            :               }
     853                 :            : 
     854                 :     670039 :           if (simple_uses < 0
     855                 :     670039 :               || all_uses != simple_uses)
     856                 :            :             {
     857                 :      16660 :               res = -1;
     858                 :      16660 :               BREAK_FROM_IMM_USE_STMT (imm_iter);
     859                 :            :             }
     860                 :     653379 :           res += all_uses;
     861                 :            :         }
     862                 :    1424600 :       else if ((is_gimple_assign (stmt) && !gimple_has_volatile_ops (stmt))
     863                 :    1446560 :                || gimple_code (stmt) == GIMPLE_PHI)
     864                 :            :         {
     865                 :    1127980 :           tree lhs;
     866                 :    1127980 :           if (gimple_code (stmt) == GIMPLE_PHI)
     867                 :     118664 :             lhs = gimple_phi_result (stmt);
     868                 :            :           else
     869                 :    1009320 :             lhs = gimple_assign_lhs (stmt);
     870                 :            : 
     871                 :    1127980 :           if (TREE_CODE (lhs) != SSA_NAME)
     872                 :            :             {
     873                 :     228656 :               res = -1;
     874                 :     228656 :               BREAK_FROM_IMM_USE_STMT (imm_iter);
     875                 :            :             }
     876                 :    1679990 :           gcc_assert (!gimple_vdef (stmt));
     877                 :     899328 :           if (bitmap_set_bit (analyzed, SSA_NAME_VERSION (lhs)))
     878                 :            :             {
     879                 :     866076 :               int tmp = isra_track_scalar_value_uses (node, lhs, parm_num,
     880                 :            :                                                       analyzed);
     881                 :     866076 :               if (tmp < 0)
     882                 :            :                 {
     883                 :     617101 :                   res = tmp;
     884                 :     617101 :                   BREAK_FROM_IMM_USE_STMT (imm_iter);
     885                 :            :                 }
     886                 :     248975 :               res += tmp;
     887                 :            :             }
     888                 :            :         }
     889                 :            :       else
     890                 :            :         {
     891                 :     296617 :           res = -1;
     892                 :    1732760 :           BREAK_FROM_IMM_USE_STMT (imm_iter);
     893                 :            :         }
     894                 :            :     }
     895                 :    1671060 :   return res;
     896                 :            : }
     897                 :            : 
     898                 :            : /* Inspect all uses of PARM, which must be a gimple register, in FUN (which is
     899                 :            :    also described by NODE) and simple arithmetic calculations involving PARM
     900                 :            :    and return false if any of them is used for something else than either an
     901                 :            :    actual call argument, simple arithmetic operation or debug statement.  If
     902                 :            :    there are no such uses, return true and store the number of actual arguments
     903                 :            :    that this parameter eventually feeds to (or zero if there is none) to
     904                 :            :    *CALL_USES_P.  For any such parameter, mark PARM_NUM as one of its
     905                 :            :    sources.
     906                 :            : 
     907                 :            :    This function is similar to ptr_parm_has_nonarg_uses but its results are
     908                 :            :    meant for unused parameter removal, as opposed to splitting of parameters
     909                 :            :    passed by reference or converting them to passed by value.
     910                 :            :   */
     911                 :            : 
     912                 :            : static bool
     913                 :    1389920 : isra_track_scalar_param_local_uses (function *fun, cgraph_node *node, tree parm,
     914                 :            :                                     int parm_num, int *call_uses_p)
     915                 :            : {
     916                 :    1389920 :   gcc_checking_assert (is_gimple_reg (parm));
     917                 :            : 
     918                 :    1389920 :   tree name = ssa_default_def (fun, parm);
     919                 :    1389920 :   if (!name || has_zero_uses (name))
     920                 :            :     {
     921                 :     584929 :       *call_uses_p = 0;
     922                 :     584929 :       return false;
     923                 :            :     }
     924                 :            : 
     925                 :            :   /* Edge summaries can only handle callers with fewer than 256 parameters.  */
     926                 :     804989 :   if (parm_num > UCHAR_MAX)
     927                 :            :     return true;
     928                 :            : 
     929                 :     804989 :   bitmap analyzed = BITMAP_ALLOC (NULL);
     930                 :     804989 :   int call_uses = isra_track_scalar_value_uses (node, name, parm_num, analyzed);
     931                 :     804989 :   BITMAP_FREE (analyzed);
     932                 :     804989 :   if (call_uses < 0)
     933                 :            :     return true;
     934                 :     257115 :   *call_uses_p = call_uses;
     935                 :     257115 :   return false;
     936                 :            : }
     937                 :            : 
     938                 :            : /* Scan immediate uses of a default definition SSA name of a parameter PARM and
     939                 :            :    examine whether there are any nonarg uses that are not actual arguments or
     940                 :            :    otherwise infeasible uses.  If so, return true, otherwise return false.
     941                 :            :    Create pass-through IPA flow records for any direct uses as argument calls
     942                 :            :    and if returning false, store their number into *PT_COUNT_P.  NODE and FUN
     943                 :            :    must represent the function that is currently analyzed, PARM_NUM must be the
     944                 :            :    index of the analyzed parameter.
     945                 :            : 
     946                 :            :    This function is similar to isra_track_scalar_param_local_uses but its
     947                 :            :    results are meant for splitting of parameters passed by reference or turning
     948                 :            :    them into bits passed by value, as opposed to generic unused parameter
     949                 :            :    removal.
     950                 :            :  */
     951                 :            : 
     952                 :            : static bool
     953                 :     612194 : ptr_parm_has_nonarg_uses (cgraph_node *node, function *fun, tree parm,
     954                 :            :                           int parm_num, unsigned *pt_count_p)
     955                 :            : {
     956                 :     612194 :   imm_use_iterator ui;
     957                 :     612194 :   gimple *stmt;
     958                 :     612194 :   tree name = ssa_default_def (fun, parm);
     959                 :     612194 :   bool ret = false;
     960                 :     612194 :   unsigned pt_count = 0;
     961                 :            : 
     962                 :     612194 :   if (!name || has_zero_uses (name))
     963                 :            :     return false;
     964                 :            : 
     965                 :            :   /* Edge summaries can only handle callers with fewer than 256 parameters.  */
     966                 :     567584 :   if (parm_num > UCHAR_MAX)
     967                 :            :     return true;
     968                 :            : 
     969                 :    1557810 :   FOR_EACH_IMM_USE_STMT (stmt, ui, name)
     970                 :            :     {
     971                 :    1314760 :       unsigned uses_ok = 0;
     972                 :    1314760 :       use_operand_p use_p;
     973                 :            : 
     974                 :    1314760 :       if (is_gimple_debug (stmt))
     975                 :     396471 :         continue;
     976                 :            : 
     977                 :     918286 :       if (gimple_assign_single_p (stmt))
     978                 :            :         {
     979                 :     518933 :           tree rhs = gimple_assign_rhs1 (stmt);
     980                 :     984169 :           while (handled_component_p (rhs))
     981                 :     465236 :             rhs = TREE_OPERAND (rhs, 0);
     982                 :     518933 :           if (TREE_CODE (rhs) == MEM_REF
     983                 :     350901 :               && TREE_OPERAND (rhs, 0) == name
     984                 :     350289 :               && integer_zerop (TREE_OPERAND (rhs, 1))
     985                 :     327681 :               && types_compatible_p (TREE_TYPE (rhs),
     986                 :     327681 :                                      TREE_TYPE (TREE_TYPE (name)))
     987                 :     807599 :               && !TREE_THIS_VOLATILE (rhs))
     988                 :            :             uses_ok++;
     989                 :            :         }
     990                 :     399353 :       else if (is_gimple_call (stmt))
     991                 :            :         {
     992                 :     313071 :           gcall *call = as_a <gcall *> (stmt);
     993                 :     313071 :           unsigned arg_count;
     994                 :     313071 :           if (gimple_call_internal_p (call)
     995                 :     313071 :               || (arg_count = gimple_call_num_args (call)) == 0)
     996                 :            :             {
     997                 :        817 :               ret = true;
     998                 :        817 :               BREAK_FROM_IMM_USE_STMT (ui);
     999                 :            :             }
    1000                 :            : 
    1001                 :     312254 :           cgraph_edge *cs = node->get_edge (stmt);
    1002                 :     312254 :           gcc_checking_assert (cs);
    1003                 :     312254 :           isra_call_summary *csum = call_sums->get_create (cs);
    1004                 :     312254 :           csum->init_inputs (arg_count);
    1005                 :            : 
    1006                 :    1269860 :           for (unsigned i = 0; i < arg_count; ++i)
    1007                 :            :             {
    1008                 :     957602 :               tree arg = gimple_call_arg (stmt, i);
    1009                 :            : 
    1010                 :     957602 :               if (arg == name)
    1011                 :            :                 {
    1012                 :            :                   /* TODO: Allow &MEM_REF[name + offset] here,
    1013                 :            :                      ipa_param_body_adjustments::modify_call_stmt has to be
    1014                 :            :                      adjusted too.  */
    1015                 :     303974 :                   csum->m_arg_flow[i].pointer_pass_through = true;
    1016                 :     303974 :                   set_single_param_flow_source (&csum->m_arg_flow[i], parm_num);
    1017                 :     303974 :                   pt_count++;
    1018                 :     303974 :                   uses_ok++;
    1019                 :     303974 :                   continue;
    1020                 :            :                 }
    1021                 :            : 
    1022                 :     673457 :               while (handled_component_p (arg))
    1023                 :      19829 :                 arg = TREE_OPERAND (arg, 0);
    1024                 :     653628 :               if (TREE_CODE (arg) == MEM_REF
    1025                 :      14686 :                   && TREE_OPERAND (arg, 0) == name
    1026                 :       7777 :                   && integer_zerop (TREE_OPERAND (arg, 1))
    1027                 :       7777 :                   && types_compatible_p (TREE_TYPE (arg),
    1028                 :       7777 :                                          TREE_TYPE (TREE_TYPE (name)))
    1029                 :     658990 :                   && !TREE_THIS_VOLATILE (arg))
    1030                 :       5362 :                 uses_ok++;
    1031                 :            :             }
    1032                 :            :         }
    1033                 :            : 
    1034                 :            :       /* If the number of valid uses does not match the number of
    1035                 :            :          uses in this stmt there is an unhandled use.  */
    1036                 :     917469 :       unsigned all_uses = 0;
    1037                 :    1840460 :       FOR_EACH_IMM_USE_ON_STMT (use_p, ui)
    1038                 :     922989 :         all_uses++;
    1039                 :            : 
    1040                 :     917469 :       gcc_checking_assert (uses_ok <= all_uses);
    1041                 :     917469 :       if (uses_ok != all_uses)
    1042                 :            :         {
    1043                 :     323718 :           ret = true;
    1044                 :    1313940 :           BREAK_FROM_IMM_USE_STMT (ui);
    1045                 :            :         }
    1046                 :            :     }
    1047                 :            : 
    1048                 :     567584 :   *pt_count_p = pt_count;
    1049                 :     567584 :   return ret;
    1050                 :            : }
    1051                 :            : 
    1052                 :            : /* Initialize vector of parameter descriptors of NODE.  Return true if there
    1053                 :            :    are any candidates for splitting or unused aggregate parameter removal (the
    1054                 :            :    function may return false if there are candidates for removal of register
    1055                 :            :    parameters) and function body must be scanned.  */
    1056                 :            : 
    1057                 :            : static bool
    1058                 :     492542 : create_parameter_descriptors (cgraph_node *node,
    1059                 :            :                               vec<gensum_param_desc> *param_descriptions)
    1060                 :            : {
    1061                 :     492542 :   function *fun = DECL_STRUCT_FUNCTION (node->decl);
    1062                 :     492542 :   bool ret = false;
    1063                 :            : 
    1064                 :     492542 :   int num = 0;
    1065                 :     492542 :   for (tree parm = DECL_ARGUMENTS (node->decl);
    1066                 :    2012600 :        parm;
    1067                 :    1520060 :        parm = DECL_CHAIN (parm), num++)
    1068                 :            :     {
    1069                 :    1520060 :       const char *msg;
    1070                 :    1520060 :       gensum_param_desc *desc = &(*param_descriptions)[num];
    1071                 :            :       /* param_descriptions vector is grown cleared in the caller.  */
    1072                 :    1520060 :       desc->param_number = num;
    1073                 :    1520060 :       decl2desc->put (parm, desc);
    1074                 :            : 
    1075                 :    1520060 :       if (dump_file && (dump_flags & TDF_DETAILS))
    1076                 :          5 :         print_generic_expr (dump_file, parm, TDF_UID);
    1077                 :            : 
    1078                 :    1520060 :       int scalar_call_uses;
    1079                 :    1520060 :       tree type = TREE_TYPE (parm);
    1080                 :    1520060 :       if (TREE_THIS_VOLATILE (parm))
    1081                 :            :         {
    1082                 :         12 :           if (dump_file && (dump_flags & TDF_DETAILS))
    1083                 :          0 :             fprintf (dump_file, " not a candidate, is volatile\n");
    1084                 :    1170630 :           continue;
    1085                 :            :         }
    1086                 :    1520050 :       if (!is_gimple_reg_type (type) && is_va_list_type (type))
    1087                 :            :         {
    1088                 :          0 :           if (dump_file && (dump_flags & TDF_DETAILS))
    1089                 :          0 :             fprintf (dump_file, " not a candidate, is a va_list type\n");
    1090                 :          0 :           continue;
    1091                 :            :         }
    1092                 :    1520050 :       if (TREE_ADDRESSABLE (parm))
    1093                 :            :         {
    1094                 :      16731 :           if (dump_file && (dump_flags & TDF_DETAILS))
    1095                 :          0 :             fprintf (dump_file, " not a candidate, is addressable\n");
    1096                 :      16731 :           continue;
    1097                 :            :         }
    1098                 :    1503320 :       if (TREE_ADDRESSABLE (type))
    1099                 :            :         {
    1100                 :          0 :           if (dump_file && (dump_flags & TDF_DETAILS))
    1101                 :          0 :             fprintf (dump_file, " not a candidate, type cannot be split\n");
    1102                 :          0 :           continue;
    1103                 :            :         }
    1104                 :            : 
    1105                 :    1503320 :       if (is_gimple_reg (parm)
    1106                 :    1503320 :           && !isra_track_scalar_param_local_uses (fun, node, parm, num,
    1107                 :            :                                                   &scalar_call_uses))
    1108                 :            :         {
    1109                 :     842044 :           if (dump_file && (dump_flags & TDF_DETAILS))
    1110                 :          4 :             fprintf (dump_file, " is a scalar with only %i call uses\n",
    1111                 :            :                      scalar_call_uses);
    1112                 :            : 
    1113                 :     842044 :           desc->locally_unused = true;
    1114                 :     842044 :           desc->call_uses = scalar_call_uses;
    1115                 :            :         }
    1116                 :            : 
    1117                 :    1503320 :       if (POINTER_TYPE_P (type))
    1118                 :            :         {
    1119                 :     616244 :           type = TREE_TYPE (type);
    1120                 :            : 
    1121                 :     618960 :           if (TREE_CODE (type) == FUNCTION_TYPE
    1122                 :     616244 :               || TREE_CODE (type) == METHOD_TYPE)
    1123                 :            :             {
    1124                 :       2716 :               if (dump_file && (dump_flags & TDF_DETAILS))
    1125                 :          0 :                 fprintf (dump_file, " not a candidate, reference to "
    1126                 :            :                          "a function\n");
    1127                 :       2716 :               continue;
    1128                 :            :             }
    1129                 :     613528 :           if (TYPE_VOLATILE (type))
    1130                 :            :             {
    1131                 :       1201 :               if (dump_file && (dump_flags & TDF_DETAILS))
    1132                 :          0 :                 fprintf (dump_file, " not a candidate, reference to "
    1133                 :            :                          "a volatile type\n");
    1134                 :       1201 :               continue;
    1135                 :            :             }
    1136                 :     612327 :           if (TREE_CODE (type) == ARRAY_TYPE
    1137                 :     612327 :               && TYPE_NONALIASED_COMPONENT (type))
    1138                 :            :             {
    1139                 :          0 :               if (dump_file && (dump_flags & TDF_DETAILS))
    1140                 :          0 :                 fprintf (dump_file, " not a candidate, reference to "
    1141                 :            :                          "a nonaliased component array\n");
    1142                 :          0 :               continue;
    1143                 :            :             }
    1144                 :     612327 :           if (!is_gimple_reg (parm))
    1145                 :            :             {
    1146                 :          0 :               if (dump_file && (dump_flags & TDF_DETAILS))
    1147                 :          0 :                 fprintf (dump_file, " not a candidate, a reference which is "
    1148                 :            :                          "not a gimple register (probably addressable)\n");
    1149                 :          0 :               continue;
    1150                 :            :             }
    1151                 :     612327 :           if (is_va_list_type (type))
    1152                 :            :             {
    1153                 :        133 :               if (dump_file && (dump_flags & TDF_DETAILS))
    1154                 :          0 :                 fprintf (dump_file, " not a candidate, reference to "
    1155                 :            :                          "a va list\n");
    1156                 :        133 :               continue;
    1157                 :            :             }
    1158                 :     612194 :           if (ptr_parm_has_nonarg_uses (node, fun, parm, num,
    1159                 :     612194 :                                         &desc->ptr_pt_count))
    1160                 :            :             {
    1161                 :     324535 :               if (dump_file && (dump_flags & TDF_DETAILS))
    1162                 :          0 :                 fprintf (dump_file, " not a candidate, reference has "
    1163                 :            :                          "nonarg uses\n");
    1164                 :     324535 :               continue;
    1165                 :            :             }
    1166                 :     287659 :           desc->by_ref = true;
    1167                 :            :         }
    1168                 :     887072 :       else if (!AGGREGATE_TYPE_P (type))
    1169                 :            :         {
    1170                 :            :           /* This is in an else branch because scalars passed by reference are
    1171                 :            :              still candidates to be passed by value.  */
    1172                 :     773781 :           if (dump_file && (dump_flags & TDF_DETAILS))
    1173                 :          2 :             fprintf (dump_file, " not a candidate, not an aggregate\n");
    1174                 :     773781 :           continue;
    1175                 :            :         }
    1176                 :            : 
    1177                 :     400950 :       if (!COMPLETE_TYPE_P (type))
    1178                 :            :         {
    1179                 :      42863 :           if (dump_file && (dump_flags & TDF_DETAILS))
    1180                 :          0 :             fprintf (dump_file, " not a candidate, not a complete type\n");
    1181                 :      42863 :           continue;
    1182                 :            :         }
    1183                 :     358087 :       if (!tree_fits_uhwi_p (TYPE_SIZE (type)))
    1184                 :            :         {
    1185                 :         76 :           if (dump_file && (dump_flags & TDF_DETAILS))
    1186                 :          0 :             fprintf (dump_file, " not a candidate, size not representable\n");
    1187                 :         76 :           continue;
    1188                 :            :         }
    1189                 :     358011 :       unsigned HOST_WIDE_INT type_size
    1190                 :     358011 :         = tree_to_uhwi (TYPE_SIZE (type)) / BITS_PER_UNIT;
    1191                 :     360085 :       if (type_size == 0
    1192                 :     358011 :           || type_size >= ISRA_ARG_SIZE_LIMIT)
    1193                 :            :         {
    1194                 :       2074 :           if (dump_file && (dump_flags & TDF_DETAILS))
    1195                 :          0 :             fprintf (dump_file, " not a candidate, has zero or huge size\n");
    1196                 :       2074 :           continue;
    1197                 :            :         }
    1198                 :     355937 :       if (type_internals_preclude_sra_p (type, &msg))
    1199                 :            :         {
    1200                 :       6512 :           if (dump_file && (dump_flags & TDF_DETAILS))
    1201                 :          0 :               fprintf (dump_file, " not a candidate, %s\n", msg);
    1202                 :       6512 :           continue;
    1203                 :            :         }
    1204                 :            : 
    1205                 :     349425 :       if (dump_file && (dump_flags & TDF_DETAILS))
    1206                 :          3 :         fprintf (dump_file, " is a candidate\n");
    1207                 :            : 
    1208                 :     349425 :       ret = true;
    1209                 :     349425 :       desc->split_candidate = true;
    1210                 :     349425 :       if (desc->by_ref)
    1211                 :     236219 :         desc->deref_index = by_ref_count++;
    1212                 :            :     }
    1213                 :     492542 :   return ret;
    1214                 :            : }
    1215                 :            : 
    1216                 :            : /* Return pointer to descriptor of parameter DECL or NULL if it cannot be
    1217                 :            :    found, which happens if DECL is for a static chain.  */
    1218                 :            : 
    1219                 :            : static gensum_param_desc *
    1220                 :     721683 : get_gensum_param_desc (tree decl)
    1221                 :            : {
    1222                 :     721683 :   gcc_checking_assert (TREE_CODE (decl) == PARM_DECL);
    1223                 :     721683 :   gensum_param_desc **slot = decl2desc->get (decl);
    1224                 :     709131 :   if (!slot)
    1225                 :            :     /* This can happen for static chains which we cannot handle so far.  */
    1226                 :            :     return NULL;
    1227                 :     709131 :   gcc_checking_assert (*slot);
    1228                 :            :   return *slot;
    1229                 :            : }
    1230                 :            : 
    1231                 :            : 
    1232                 :            : /* Remove parameter described by DESC from candidates for IPA-SRA splitting and
    1233                 :            :    write REASON to the dump file if there is one.  */
    1234                 :            : 
    1235                 :            : static void
    1236                 :     121255 : disqualify_split_candidate (gensum_param_desc *desc, const char *reason)
    1237                 :            : {
    1238                 :     121255 :   if (!desc->split_candidate)
    1239                 :            :     return;
    1240                 :            : 
    1241                 :     121255 :   if (dump_file && (dump_flags & TDF_DETAILS))
    1242                 :          0 :     fprintf (dump_file, "! Disqualifying parameter number %i - %s\n",
    1243                 :            :              desc->param_number, reason);
    1244                 :            : 
    1245                 :     121255 :   desc->split_candidate = false;
    1246                 :            : }
    1247                 :            : 
    1248                 :            : /* Remove DECL from candidates for IPA-SRA and write REASON to the dump file if
    1249                 :            :    there is one.  */
    1250                 :            : 
    1251                 :            : static void
    1252                 :          0 : disqualify_split_candidate (tree decl, const char *reason)
    1253                 :            : {
    1254                 :          0 :   gensum_param_desc *desc = get_gensum_param_desc (decl);
    1255                 :          0 :   if (desc)
    1256                 :          0 :     disqualify_split_candidate (desc, reason);
    1257                 :          0 : }
    1258                 :            : 
    1259                 :            : /* Allocate a new access to DESC and fill it in with OFFSET and SIZE.  But
    1260                 :            :    first, check that there are not too many of them already.  If so, do not
    1261                 :            :    allocate anything and return NULL.  */
    1262                 :            : 
    1263                 :            : static gensum_param_access *
    1264                 :     255923 : allocate_access (gensum_param_desc *desc,
    1265                 :            :                  HOST_WIDE_INT offset, HOST_WIDE_INT size)
    1266                 :            : {
    1267                 :     255923 :   if (desc->access_count
    1268                 :     255923 :       == (unsigned) param_ipa_sra_max_replacements)
    1269                 :            :     {
    1270                 :          0 :       disqualify_split_candidate (desc, "Too many replacement candidates");
    1271                 :          0 :       return NULL;
    1272                 :            :     }
    1273                 :            : 
    1274                 :     255923 :   gensum_param_access *access
    1275                 :     255923 :     = (gensum_param_access *) obstack_alloc (&gensum_obstack,
    1276                 :            :                                              sizeof (gensum_param_access));
    1277                 :     255923 :   memset (access, 0, sizeof (*access));
    1278                 :     255923 :   access->offset = offset;
    1279                 :     255923 :   access->size = size;
    1280                 :     255923 :   return access;
    1281                 :            : }
    1282                 :            : 
    1283                 :            : /* In what context scan_expr_access has been called, whether it deals with a
    1284                 :            :    load, a function argument, or a store.  */
    1285                 :            : 
    1286                 :            : enum isra_scan_context {ISRA_CTX_LOAD, ISRA_CTX_ARG, ISRA_CTX_STORE};
    1287                 :            : 
    1288                 :            : /* Return an access describing memory access to the variable described by DESC
    1289                 :            :    at OFFSET with SIZE in context CTX, starting at pointer to the linked list
    1290                 :            :    at a certain tree level FIRST.  Attempt to create it and put into the
    1291                 :            :    appropriate place in the access tree if does not exist, but fail and return
    1292                 :            :    NULL if there are already too many accesses, if it would create a partially
    1293                 :            :    overlapping access or if an access would end up within a pre-existing
    1294                 :            :    non-call access.  */
    1295                 :            : 
    1296                 :            : static gensum_param_access *
    1297                 :     299035 : get_access_1 (gensum_param_desc *desc, gensum_param_access **first,
    1298                 :            :               HOST_WIDE_INT offset, HOST_WIDE_INT size, isra_scan_context ctx)
    1299                 :            : {
    1300                 :     306451 :   gensum_param_access *access = *first, **ptr = first;
    1301                 :            : 
    1302                 :     306451 :   if (!access)
    1303                 :            :     {
    1304                 :            :       /* No pre-existing access at this level, just create it.  */
    1305                 :     165968 :       gensum_param_access *a = allocate_access (desc, offset, size);
    1306                 :     165968 :       if (!a)
    1307                 :            :         return NULL;
    1308                 :     165968 :       *first = a;
    1309                 :     165968 :       return *first;
    1310                 :            :     }
    1311                 :            : 
    1312                 :     140483 :   if (access->offset >= offset + size)
    1313                 :            :     {
    1314                 :            :       /* We want to squeeze it in front of the very first access, just do
    1315                 :            :          it.  */
    1316                 :      21490 :       gensum_param_access *r = allocate_access (desc, offset, size);
    1317                 :      21490 :       if (!r)
    1318                 :            :         return NULL;
    1319                 :      21490 :       r->next_sibling = access;
    1320                 :      21490 :       *first = r;
    1321                 :      21490 :       return r;
    1322                 :            :     }
    1323                 :            : 
    1324                 :            :   /* Skip all accesses that have to come before us until the next sibling is
    1325                 :            :      already too far.  */
    1326                 :     169224 :   while (offset >= access->offset + access->size
    1327                 :     110257 :          && access->next_sibling
    1328                 :     223661 :          && access->next_sibling->offset < offset + size)
    1329                 :            :     {
    1330                 :      50231 :       ptr = &access->next_sibling;
    1331                 :      50231 :       access = access->next_sibling;
    1332                 :            :     }
    1333                 :            : 
    1334                 :            :   /* At this point we know we do not belong before access.  */
    1335                 :     118993 :   gcc_assert (access->offset < offset + size);
    1336                 :            : 
    1337                 :     118993 :   if (access->offset == offset && access->size == size)
    1338                 :            :     /* We found what we were looking for.  */
    1339                 :            :     return access;
    1340                 :            : 
    1341                 :      79968 :   if (access->offset <= offset
    1342                 :      77179 :       && access->offset + access->size >= offset + size)
    1343                 :            :     {
    1344                 :            :     /* We fit into access which is larger than us.  We need to find/create
    1345                 :            :        something below access.  But we only allow nesting in call
    1346                 :            :        arguments.  */
    1347                 :       8989 :       if (access->nonarg)
    1348                 :            :         return NULL;
    1349                 :            : 
    1350                 :       7416 :       return get_access_1 (desc, &access->first_child, offset, size, ctx);
    1351                 :            :     }
    1352                 :            : 
    1353                 :      70979 :   if (offset <= access->offset
    1354                 :      10953 :       && offset + size  >= access->offset + access->size)
    1355                 :            :     /* We are actually bigger than access, which fully fits into us, take its
    1356                 :            :        place and make all accesses fitting into it its children.  */
    1357                 :            :     {
    1358                 :            :       /* But first, we only allow nesting in call arguments so check if that is
    1359                 :            :          what we are trying to represent.  */
    1360                 :      10953 :       if (ctx != ISRA_CTX_ARG)
    1361                 :            :         return NULL;
    1362                 :            : 
    1363                 :       8439 :       gensum_param_access *r = allocate_access (desc, offset, size);
    1364                 :       8439 :       if (!r)
    1365                 :            :         return NULL;
    1366                 :       8439 :       r->first_child = access;
    1367                 :            : 
    1368                 :      15544 :       while (access->next_sibling
    1369                 :      15544 :              && access->next_sibling->offset < offset + size)
    1370                 :            :         access = access->next_sibling;
    1371                 :       8439 :       if (access->offset + access->size > offset + size)
    1372                 :            :         {
    1373                 :            :           /* This must be a different access, which are sorted, so the
    1374                 :            :              following must be true and this signals a partial overlap.  */
    1375                 :          0 :           gcc_assert (access->offset > offset);
    1376                 :            :           return NULL;
    1377                 :            :         }
    1378                 :            : 
    1379                 :       8439 :       r->next_sibling = access->next_sibling;
    1380                 :       8439 :       access->next_sibling = NULL;
    1381                 :       8439 :       *ptr = r;
    1382                 :       8439 :       return r;
    1383                 :            :     }
    1384                 :            : 
    1385                 :      60026 :   if (offset >= access->offset + access->size)
    1386                 :            :     {
    1387                 :            :       /* We belong after access.  */
    1388                 :      60026 :       gensum_param_access *r = allocate_access (desc, offset, size);
    1389                 :      60026 :       if (!r)
    1390                 :            :         return NULL;
    1391                 :      60026 :       r->next_sibling = access->next_sibling;
    1392                 :      60026 :       access->next_sibling = r;
    1393                 :      60026 :       return r;
    1394                 :            :     }
    1395                 :            : 
    1396                 :          0 :   if (offset < access->offset)
    1397                 :            :     {
    1398                 :            :       /* We know the following, otherwise we would have created a
    1399                 :            :          super-access. */
    1400                 :          0 :       gcc_checking_assert (offset + size < access->offset + access->size);
    1401                 :            :       return NULL;
    1402                 :            :     }
    1403                 :            : 
    1404                 :          0 :   if (offset + size > access->offset + access->size)
    1405                 :            :     {
    1406                 :            :       /* Likewise.  */
    1407                 :          0 :       gcc_checking_assert (offset > access->offset);
    1408                 :            :       return NULL;
    1409                 :            :     }
    1410                 :            : 
    1411                 :          0 :   gcc_unreachable ();
    1412                 :            : }
    1413                 :            : 
    1414                 :            : /* Return an access describing memory access to the variable described by DESC
    1415                 :            :    at OFFSET with SIZE in context CTX, mark it as used in context CTX.  Attempt
    1416                 :            :    to create if it does not exist, but fail and return NULL if there are
    1417                 :            :    already too many accesses, if it would create a partially overlapping access
    1418                 :            :    or if an access would end up in a non-call access.  */
    1419                 :            : 
    1420                 :            : static gensum_param_access *
    1421                 :     299035 : get_access (gensum_param_desc *desc, HOST_WIDE_INT offset, HOST_WIDE_INT size,
    1422                 :            :             isra_scan_context ctx)
    1423                 :            : {
    1424                 :     299035 :   gcc_checking_assert (desc->split_candidate);
    1425                 :            : 
    1426                 :     299035 :   gensum_param_access *access = get_access_1 (desc, &desc->accesses, offset,
    1427                 :            :                                               size, ctx);
    1428                 :     299035 :   if (!access)
    1429                 :            :     {
    1430                 :       4087 :       disqualify_split_candidate (desc,
    1431                 :            :                                   "Bad access overlap or too many accesses");
    1432                 :       4087 :       return NULL;
    1433                 :            :     }
    1434                 :            : 
    1435                 :     294948 :   switch (ctx)
    1436                 :            :     {
    1437                 :      10039 :     case ISRA_CTX_STORE:
    1438                 :      10039 :       gcc_assert (!desc->by_ref);
    1439                 :            :       /* Fall-through */
    1440                 :     235536 :     case ISRA_CTX_LOAD:
    1441                 :     235536 :       access->nonarg = true;
    1442                 :     235536 :       break;
    1443                 :            :     case ISRA_CTX_ARG:
    1444                 :            :       break;
    1445                 :            :     }
    1446                 :            : 
    1447                 :            :   return access;
    1448                 :            : }
    1449                 :            : 
    1450                 :            : /* Verify that parameter access tree starting with ACCESS is in good shape.
    1451                 :            :    PARENT_OFFSET and PARENT_SIZE are the corresponding fields of parent of
    1452                 :            :    ACCESS or zero if there is none.  */
    1453                 :            : 
    1454                 :            : static bool
    1455                 :     530905 : verify_access_tree_1 (gensum_param_access *access, HOST_WIDE_INT parent_offset,
    1456                 :            :                       HOST_WIDE_INT parent_size)
    1457                 :            : {
    1458                 :     758954 :   while (access)
    1459                 :            :     {
    1460                 :     228049 :       gcc_assert (access->offset >= 0 && access->size > 0);
    1461                 :            : 
    1462                 :     228049 :       if (parent_size != 0)
    1463                 :            :         {
    1464                 :      17152 :           if (access->offset < parent_offset)
    1465                 :            :             {
    1466                 :          0 :               error ("Access offset before parent offset");
    1467                 :          0 :               return true;
    1468                 :            :             }
    1469                 :      17152 :           if (access->size >= parent_size)
    1470                 :            :             {
    1471                 :          0 :               error ("Access size greater or equal to its parent size");
    1472                 :          0 :               return true;
    1473                 :            :             }
    1474                 :      17152 :           if (access->offset + access->size > parent_offset + parent_size)
    1475                 :            :             {
    1476                 :          0 :               error ("Access terminates outside of its parent");
    1477                 :          0 :               return true;
    1478                 :            :             }
    1479                 :            :         }
    1480                 :            : 
    1481                 :     228049 :       if (verify_access_tree_1 (access->first_child, access->offset,
    1482                 :            :                                 access->size))
    1483                 :            :         return true;
    1484                 :            : 
    1485                 :     228049 :       if (access->next_sibling
    1486                 :      72903 :           && (access->next_sibling->offset < access->offset + access->size))
    1487                 :            :         {
    1488                 :          0 :           error ("Access overlaps with its sibling");
    1489                 :          0 :           return true;
    1490                 :            :         }
    1491                 :            : 
    1492                 :            :       access = access->next_sibling;
    1493                 :            :     }
    1494                 :            :   return false;
    1495                 :            : }
    1496                 :            : 
    1497                 :            : /* Verify that parameter access tree starting with ACCESS is in good shape,
    1498                 :            :    halt compilation and dump the tree to stderr if not.  */
    1499                 :            : 
    1500                 :            : DEBUG_FUNCTION void
    1501                 :     302856 : isra_verify_access_tree (gensum_param_access *access)
    1502                 :            : {
    1503                 :     302856 :   if (verify_access_tree_1 (access, 0, 0))
    1504                 :            :     {
    1505                 :          0 :       for (; access; access = access->next_sibling)
    1506                 :          0 :         dump_gensum_access (stderr, access, 2);
    1507                 :          0 :       internal_error ("IPA-SRA access verification failed");
    1508                 :            :     }
    1509                 :     302856 : }
    1510                 :            : 
    1511                 :            : 
    1512                 :            : /* Callback of walk_stmt_load_store_addr_ops visit_addr used to determine
    1513                 :            :    GIMPLE_ASM operands with memory constrains which cannot be scalarized.  */
    1514                 :            : 
    1515                 :            : static bool
    1516                 :        156 : asm_visit_addr (gimple *, tree op, tree, void *)
    1517                 :            : {
    1518                 :        156 :   op = get_base_address (op);
    1519                 :        156 :   if (op
    1520                 :        156 :       && TREE_CODE (op) == PARM_DECL)
    1521                 :          0 :     disqualify_split_candidate (op, "Non-scalarizable GIMPLE_ASM operand.");
    1522                 :            : 
    1523                 :        156 :   return false;
    1524                 :            : }
    1525                 :            : 
    1526                 :            : /* Mark a dereference of parameter identified by DESC of distance DIST in a
    1527                 :            :    basic block BB, unless the BB has already been marked as a potentially
    1528                 :            :    final.  */
    1529                 :            : 
    1530                 :            : static void
    1531                 :     117158 : mark_param_dereference (gensum_param_desc *desc, HOST_WIDE_INT dist,
    1532                 :            :                        basic_block bb)
    1533                 :            : {
    1534                 :     117158 :   gcc_assert (desc->by_ref);
    1535                 :     117158 :   gcc_checking_assert (desc->split_candidate);
    1536                 :            : 
    1537                 :     117158 :   if (bitmap_bit_p (final_bbs, bb->index))
    1538                 :            :     return;
    1539                 :            : 
    1540                 :      89658 :   int idx = bb->index * by_ref_count + desc->deref_index;
    1541                 :      89658 :   if (bb_dereferences[idx] < dist)
    1542                 :      78328 :     bb_dereferences[idx] = dist;
    1543                 :            : }
    1544                 :            : 
    1545                 :            : /* Return true, if any potential replacements should use NEW_TYPE as opposed to
    1546                 :            :    previously recorded OLD_TYPE.  */
    1547                 :            : 
    1548                 :            : static bool
    1549                 :      35029 : type_prevails_p (tree old_type, tree new_type)
    1550                 :            : {
    1551                 :      35029 :   if (old_type == new_type)
    1552                 :            :     return false;
    1553                 :            : 
    1554                 :            :   /* Non-aggregates are always better.  */
    1555                 :         37 :   if (!is_gimple_reg_type (old_type)
    1556                 :         37 :       && is_gimple_reg_type (new_type))
    1557                 :            :     return true;
    1558                 :         37 :   if (is_gimple_reg_type (old_type)
    1559                 :         37 :       && !is_gimple_reg_type (new_type))
    1560                 :            :     return false;
    1561                 :            : 
    1562                 :            :   /* Prefer any complex or vector type over any other scalar type.  */
    1563                 :         37 :   if (TREE_CODE (old_type) != COMPLEX_TYPE
    1564                 :         37 :       && TREE_CODE (old_type) != VECTOR_TYPE
    1565                 :         24 :       && (TREE_CODE (new_type) == COMPLEX_TYPE
    1566                 :         24 :           || TREE_CODE (new_type) == VECTOR_TYPE))
    1567                 :            :     return true;
    1568                 :         37 :   if ((TREE_CODE (old_type) == COMPLEX_TYPE
    1569                 :            :        || TREE_CODE (old_type) == VECTOR_TYPE)
    1570                 :         13 :       && TREE_CODE (new_type) != COMPLEX_TYPE
    1571                 :          0 :       && TREE_CODE (new_type) != VECTOR_TYPE)
    1572                 :            :     return false;
    1573                 :            : 
    1574                 :            :   /* Use the integral type with the bigger precision.  */
    1575                 :         37 :   if (INTEGRAL_TYPE_P (old_type)
    1576                 :          9 :       && INTEGRAL_TYPE_P (new_type))
    1577                 :          9 :     return (TYPE_PRECISION (new_type) > TYPE_PRECISION (old_type));
    1578                 :            : 
    1579                 :            :   /* Attempt to disregard any integral type with non-full precision.  */
    1580                 :         28 :   if (INTEGRAL_TYPE_P (old_type)
    1581                 :         28 :       && (TREE_INT_CST_LOW (TYPE_SIZE (old_type))
    1582                 :          0 :           != TYPE_PRECISION (old_type)))
    1583                 :            :     return true;
    1584                 :         28 :   if (INTEGRAL_TYPE_P (new_type)
    1585                 :         28 :       && (TREE_INT_CST_LOW (TYPE_SIZE (new_type))
    1586                 :          0 :           != TYPE_PRECISION (new_type)))
    1587                 :            :     return false;
    1588                 :            :   /* Stabilize the selection.  */
    1589                 :         28 :   return TYPE_UID (old_type) < TYPE_UID (new_type);
    1590                 :            : }
    1591                 :            : 
    1592                 :            : /* When scanning an expression which is a call argument, this structure
    1593                 :            :    specifies the call and the position of the argument.  */
    1594                 :            : 
    1595                 :            : struct scan_call_info
    1596                 :            : {
    1597                 :            :   /* Call graph edge representing the call. */
    1598                 :            :   cgraph_edge *cs;
    1599                 :            :   /* Total number of arguments in the call.  */
    1600                 :            :   unsigned argument_count;
    1601                 :            :   /* Number of the actual argument being scanned.  */
    1602                 :            :   unsigned arg_idx;
    1603                 :            : };
    1604                 :            : 
    1605                 :            : /* Record use of ACCESS which belongs to a parameter described by DESC in a
    1606                 :            :    call argument described by CALL_INFO.  */
    1607                 :            : 
    1608                 :            : static void
    1609                 :      57249 : record_nonregister_call_use (gensum_param_desc *desc,
    1610                 :            :                              scan_call_info *call_info,
    1611                 :            :                              unsigned unit_offset, unsigned unit_size)
    1612                 :            : {
    1613                 :      57249 :   isra_call_summary *csum = call_sums->get_create (call_info->cs);
    1614                 :      57249 :   csum->init_inputs (call_info->argument_count);
    1615                 :            : 
    1616                 :      57249 :   isra_param_flow *param_flow = &csum->m_arg_flow[call_info->arg_idx];
    1617                 :      57249 :   param_flow->aggregate_pass_through = true;
    1618                 :      57249 :   set_single_param_flow_source (param_flow, desc->param_number);
    1619                 :      57249 :   param_flow->unit_offset = unit_offset;
    1620                 :      57249 :   param_flow->unit_size = unit_size;
    1621                 :      57249 :   desc->call_uses++;
    1622                 :      57249 : }
    1623                 :            : 
    1624                 :            : /* Callback of walk_aliased_vdefs, just mark that there was a possible
    1625                 :            :    modification. */
    1626                 :            : 
    1627                 :            : static bool
    1628                 :      42501 : mark_maybe_modified (ao_ref *, tree, void *data)
    1629                 :            : {
    1630                 :      42501 :   bool *maybe_modified = (bool *) data;
    1631                 :      42501 :   *maybe_modified = true;
    1632                 :      42501 :   return true;
    1633                 :            : }
    1634                 :            : 
    1635                 :            : /* Analyze expression EXPR from GIMPLE for accesses to parameters. CTX
    1636                 :            :    specifies whether EXPR is used in a load, store or as an argument call. BB
    1637                 :            :    must be the basic block in which expr resides.  If CTX specifies call
    1638                 :            :    argument context, CALL_INFO must describe that call and argument position,
    1639                 :            :    otherwise it is ignored.  */
    1640                 :            : 
    1641                 :            : static void
    1642                 :    9685280 : scan_expr_access (tree expr, gimple *stmt, isra_scan_context ctx,
    1643                 :            :                   basic_block bb, scan_call_info *call_info = NULL)
    1644                 :            : {
    1645                 :    9685280 :   poly_int64 poffset, psize, pmax_size;
    1646                 :    9685280 :   HOST_WIDE_INT offset, size, max_size;
    1647                 :    9685280 :   tree base;
    1648                 :    9685280 :   bool deref = false;
    1649                 :    9685280 :   bool reverse;
    1650                 :            : 
    1651                 :    9685280 :   if (TREE_CODE (expr) == BIT_FIELD_REF
    1652                 :    9685280 :       || TREE_CODE (expr) == IMAGPART_EXPR
    1653                 :    9682930 :       || TREE_CODE (expr) == REALPART_EXPR)
    1654                 :       3009 :     expr = TREE_OPERAND (expr, 0);
    1655                 :            : 
    1656                 :    9685280 :   base = get_ref_base_and_extent (expr, &poffset, &psize, &pmax_size, &reverse);
    1657                 :            : 
    1658                 :    9685280 :   if (TREE_CODE (base) == MEM_REF)
    1659                 :            :     {
    1660                 :    1108300 :       tree op = TREE_OPERAND (base, 0);
    1661                 :    1108300 :       if (TREE_CODE (op) != SSA_NAME
    1662                 :    1108300 :           || !SSA_NAME_IS_DEFAULT_DEF (op))
    1663                 :    9394330 :         return;
    1664                 :    9901300 :       base = SSA_NAME_VAR (op);
    1665                 :            :       if (!base)
    1666                 :            :         return;
    1667                 :            :       deref = true;
    1668                 :            :     }
    1669                 :    9083950 :   if (TREE_CODE (base) != PARM_DECL)
    1670                 :            :     return;
    1671                 :            : 
    1672                 :     721683 :   gensum_param_desc *desc = get_gensum_param_desc (base);
    1673                 :     721683 :   if (!desc || !desc->split_candidate)
    1674                 :            :     return;
    1675                 :            : 
    1676                 :     337521 :   if (!poffset.is_constant (&offset)
    1677                 :     337521 :       || !psize.is_constant (&size)
    1678                 :     337521 :       || !pmax_size.is_constant (&max_size))
    1679                 :            :     {
    1680                 :            :       disqualify_split_candidate (desc, "Encountered a polynomial-sized "
    1681                 :            :                                   "access.");
    1682                 :            :       return;
    1683                 :            :     }
    1684                 :     337521 :   if (size < 0 || size != max_size)
    1685                 :            :     {
    1686                 :       2992 :       disqualify_split_candidate (desc, "Encountered a variable sized access.");
    1687                 :       2992 :       return;
    1688                 :            :     }
    1689                 :     334529 :   if (TREE_CODE (expr) == COMPONENT_REF
    1690                 :     334529 :       && DECL_BIT_FIELD (TREE_OPERAND (expr, 1)))
    1691                 :            :     {
    1692                 :        239 :       disqualify_split_candidate (desc, "Encountered a bit-field access.");
    1693                 :        239 :       return;
    1694                 :            :     }
    1695                 :     334290 :   if (offset < 0)
    1696                 :            :     {
    1697                 :          1 :       disqualify_split_candidate (desc, "Encountered an access at a "
    1698                 :            :                                   "negative offset.");
    1699                 :          1 :       return;
    1700                 :            :     }
    1701                 :     334289 :   gcc_assert ((offset % BITS_PER_UNIT) == 0);
    1702                 :     334289 :   gcc_assert ((size % BITS_PER_UNIT) == 0);
    1703                 :     334289 :   if ((offset / BITS_PER_UNIT) >= (UINT_MAX - ISRA_ARG_SIZE_LIMIT)
    1704                 :     334289 :       || (size / BITS_PER_UNIT) >= ISRA_ARG_SIZE_LIMIT)
    1705                 :            :     {
    1706                 :          0 :       disqualify_split_candidate (desc, "Encountered an access with too big "
    1707                 :            :                                   "offset or size");
    1708                 :          0 :       return;
    1709                 :            :     }
    1710                 :            : 
    1711                 :     334289 :   tree type = TREE_TYPE (expr);
    1712                 :     334289 :   unsigned int exp_align = get_object_alignment (expr);
    1713                 :            : 
    1714                 :     334289 :   if (exp_align < TYPE_ALIGN (type))
    1715                 :            :     {
    1716                 :        847 :       disqualify_split_candidate (desc, "Underaligned access.");
    1717                 :        847 :       return;
    1718                 :            :     }
    1719                 :            : 
    1720                 :     333442 :   if (deref)
    1721                 :            :     {
    1722                 :     151565 :       if (!desc->by_ref)
    1723                 :            :         {
    1724                 :          0 :           disqualify_split_candidate (desc, "Dereferencing a non-reference.");
    1725                 :      34407 :           return;
    1726                 :            :         }
    1727                 :     151565 :       else if (ctx == ISRA_CTX_STORE)
    1728                 :            :         {
    1729                 :          0 :           disqualify_split_candidate (desc, "Storing to data passed by "
    1730                 :            :                                       "reference.");
    1731                 :          0 :           return;
    1732                 :            :         }
    1733                 :            : 
    1734                 :     151565 :       if (!aa_walking_limit)
    1735                 :            :         {
    1736                 :          0 :           disqualify_split_candidate (desc, "Out of alias analysis step "
    1737                 :            :                                       "limit.");
    1738                 :          0 :           return;
    1739                 :            :         }
    1740                 :            : 
    1741                 :     303130 :       gcc_checking_assert (gimple_vuse (stmt));
    1742                 :     151565 :       bool maybe_modified = false;
    1743                 :     151565 :       ao_ref ar;
    1744                 :            : 
    1745                 :     151565 :       ao_ref_init (&ar, expr);
    1746                 :     151565 :       bitmap visited = BITMAP_ALLOC (NULL);
    1747                 :     303130 :       int walked = walk_aliased_vdefs (&ar, gimple_vuse (stmt),
    1748                 :            :                                        mark_maybe_modified, &maybe_modified,
    1749                 :            :                                        &visited, NULL, aa_walking_limit);
    1750                 :     151565 :       BITMAP_FREE (visited);
    1751                 :     151565 :       if (walked > 0)
    1752                 :            :         {
    1753                 :      59185 :           gcc_assert (aa_walking_limit > walked);
    1754                 :      59185 :           aa_walking_limit = aa_walking_limit - walked;
    1755                 :            :         }
    1756                 :     151565 :       if (walked < 0)
    1757                 :          0 :         aa_walking_limit = 0;
    1758                 :     151565 :       if (maybe_modified || walked < 0)
    1759                 :            :         {
    1760                 :      34407 :           disqualify_split_candidate (desc, "Data passed by reference possibly "
    1761                 :            :                                       "modified through an alias.");
    1762                 :      34407 :           return;
    1763                 :            :         }
    1764                 :            :       else
    1765                 :     117158 :         mark_param_dereference (desc, offset + size, bb);
    1766                 :            :     }
    1767                 :            :   else
    1768                 :            :     /* Pointer parameters with direct uses should have been ruled out by
    1769                 :            :        analyzing SSA default def when looking at the parameters.  */
    1770                 :     181877 :     gcc_assert (!desc->by_ref);
    1771                 :            : 
    1772                 :     299035 :   gensum_param_access *access = get_access (desc, offset, size, ctx);
    1773                 :     299035 :   if (!access)
    1774                 :            :     return;
    1775                 :            : 
    1776                 :     294948 :   if (ctx == ISRA_CTX_ARG)
    1777                 :            :     {
    1778                 :      59412 :       gcc_checking_assert (call_info);
    1779                 :            : 
    1780                 :      59412 :       if (!deref)
    1781                 :      57249 :         record_nonregister_call_use (desc, call_info, offset / BITS_PER_UNIT,
    1782                 :      57249 :                                      size / BITS_PER_UNIT);
    1783                 :            :       else
    1784                 :            :         /* This is not a pass-through of a pointer, this is a use like any
    1785                 :            :            other.  */
    1786                 :       2163 :         access->nonarg = true;
    1787                 :            :     }
    1788                 :            : 
    1789                 :     294948 :   if (!access->type)
    1790                 :            :     {
    1791                 :     255923 :       access->type = type;
    1792                 :     255923 :       access->alias_ptr_type = reference_alias_ptr_type (expr);
    1793                 :     255923 :       access->reverse = reverse;
    1794                 :            :     }
    1795                 :            :   else
    1796                 :            :     {
    1797                 :      39025 :       if (exp_align < TYPE_ALIGN (access->type))
    1798                 :            :         {
    1799                 :          0 :           disqualify_split_candidate (desc, "Reference has lower alignment "
    1800                 :            :                                       "than a previous one.");
    1801                 :          0 :           return;
    1802                 :            :         }
    1803                 :      39025 :       if (access->alias_ptr_type != reference_alias_ptr_type (expr))
    1804                 :            :         {
    1805                 :       1910 :           disqualify_split_candidate (desc, "Multiple alias pointer types.");
    1806                 :       1910 :           return;
    1807                 :            :         }
    1808                 :      37115 :       if (access->reverse != reverse)
    1809                 :            :         {
    1810                 :          0 :           disqualify_split_candidate (desc, "Both normal and reverse "
    1811                 :            :                                       "scalar storage order.");
    1812                 :          0 :           return;
    1813                 :            :         }
    1814                 :      37115 :       if (!deref
    1815                 :      33315 :           && (AGGREGATE_TYPE_P (type) || AGGREGATE_TYPE_P (access->type))
    1816                 :      60724 :           && (TYPE_MAIN_VARIANT (access->type) != TYPE_MAIN_VARIANT (type)))
    1817                 :            :         {
    1818                 :            :           /* We need the same aggregate type on all accesses to be able to
    1819                 :            :              distinguish transformation spots from pass-through arguments in
    1820                 :            :              the transformation phase.  */
    1821                 :       2086 :           disqualify_split_candidate (desc, "We do not support aggregate "
    1822                 :            :                                       "type punning.");
    1823                 :       2086 :           return;
    1824                 :            :         }
    1825                 :            : 
    1826                 :      35029 :       if (type_prevails_p (access->type, type))
    1827                 :         23 :          access->type = type;
    1828                 :            :     }
    1829                 :            : }
    1830                 :            : 
    1831                 :            : /* Scan body function described by NODE and FUN and create access trees for
    1832                 :            :    parameters.  */
    1833                 :            : 
    1834                 :            : static void
    1835                 :     244588 : scan_function (cgraph_node *node, struct function *fun)
    1836                 :            : {
    1837                 :     244588 :   basic_block bb;
    1838                 :            : 
    1839                 :    2455190 :   FOR_EACH_BB_FN (bb, fun)
    1840                 :            :     {
    1841                 :    2210600 :       gimple_stmt_iterator gsi;
    1842                 :   18692600 :       for (gsi = gsi_start_bb (bb); !gsi_end_p (gsi); gsi_next (&gsi))
    1843                 :            :         {
    1844                 :   14271400 :           gimple *stmt = gsi_stmt (gsi);
    1845                 :            : 
    1846                 :   14271400 :           if (stmt_can_throw_external (fun, stmt))
    1847                 :    1001840 :             bitmap_set_bit (final_bbs, bb->index);
    1848                 :   14271400 :           switch (gimple_code (stmt))
    1849                 :            :             {
    1850                 :     242207 :             case GIMPLE_RETURN:
    1851                 :     242207 :               {
    1852                 :     242207 :                 tree t = gimple_return_retval (as_a <greturn *> (stmt));
    1853                 :     242207 :                 if (t != NULL_TREE)
    1854                 :     142209 :                   scan_expr_access (t, stmt, ISRA_CTX_LOAD, bb);
    1855                 :     242207 :                 bitmap_set_bit (final_bbs, bb->index);
    1856                 :            :               }
    1857                 :     242207 :               break;
    1858                 :            : 
    1859                 :    4657160 :             case GIMPLE_ASSIGN:
    1860                 :   18928600 :               if (gimple_assign_single_p (stmt)
    1861                 :    3487590 :                   && !gimple_clobber_p (stmt))
    1862                 :            :                 {
    1863                 :    3239100 :                   tree rhs = gimple_assign_rhs1 (stmt);
    1864                 :    3239100 :                   scan_expr_access (rhs, stmt, ISRA_CTX_LOAD, bb);
    1865                 :    3239100 :                   tree lhs = gimple_assign_lhs (stmt);
    1866                 :    3239100 :                   scan_expr_access (lhs, stmt, ISRA_CTX_STORE, bb);
    1867                 :            :                 }
    1868                 :            :               break;
    1869                 :            : 
    1870                 :    1193610 :             case GIMPLE_CALL:
    1871                 :    1193610 :               {
    1872                 :    1193610 :                 unsigned argument_count = gimple_call_num_args (stmt);
    1873                 :    1193610 :                 scan_call_info call_info;
    1874                 :    1193610 :                 call_info.cs = node->get_edge (stmt);
    1875                 :    1193610 :                 call_info.argument_count = argument_count;
    1876                 :            : 
    1877                 :    3670810 :                 for (unsigned i = 0; i < argument_count; i++)
    1878                 :            :                   {
    1879                 :    2477190 :                     call_info.arg_idx = i;
    1880                 :    2477190 :                     scan_expr_access (gimple_call_arg (stmt, i), stmt,
    1881                 :            :                                       ISRA_CTX_ARG, bb, &call_info);
    1882                 :            :                   }
    1883                 :            : 
    1884                 :    1193610 :                 tree lhs = gimple_call_lhs (stmt);
    1885                 :    1193610 :                 if (lhs)
    1886                 :     586671 :                   scan_expr_access (lhs, stmt, ISRA_CTX_STORE, bb);
    1887                 :    1193610 :                 int flags = gimple_call_flags (stmt);
    1888                 :    1193610 :                 if ((flags & (ECF_CONST | ECF_PURE)) == 0)
    1889                 :    1130740 :                   bitmap_set_bit (final_bbs, bb->index);
    1890                 :            :               }
    1891                 :    1193610 :               break;
    1892                 :            : 
    1893                 :        868 :             case GIMPLE_ASM:
    1894                 :        868 :               {
    1895                 :        868 :                 gasm *asm_stmt = as_a <gasm *> (stmt);
    1896                 :        868 :                 walk_stmt_load_store_addr_ops (asm_stmt, NULL, NULL, NULL,
    1897                 :            :                                                asm_visit_addr);
    1898                 :        868 :                 bitmap_set_bit (final_bbs, bb->index);
    1899                 :            : 
    1900                 :       1291 :                 for (unsigned i = 0; i < gimple_asm_ninputs (asm_stmt); i++)
    1901                 :            :                   {
    1902                 :        423 :                     tree t = TREE_VALUE (gimple_asm_input_op (asm_stmt, i));
    1903                 :        423 :                     scan_expr_access (t, stmt, ISRA_CTX_LOAD, bb);
    1904                 :            :                   }
    1905                 :       1447 :                 for (unsigned i = 0; i < gimple_asm_noutputs (asm_stmt); i++)
    1906                 :            :                   {
    1907                 :        579 :                     tree t = TREE_VALUE (gimple_asm_output_op (asm_stmt, i));
    1908                 :        579 :                     scan_expr_access (t, stmt, ISRA_CTX_STORE, bb);
    1909                 :            :                   }
    1910                 :            :               }
    1911                 :            :               break;
    1912                 :            : 
    1913                 :            :             default:
    1914                 :            :               break;
    1915                 :            :             }
    1916                 :            :         }
    1917                 :            :     }
    1918                 :     244588 : }
    1919                 :            : 
    1920                 :            : /* Return true if SSA_NAME NAME is only used in return statements, or if
    1921                 :            :    results of any operations it is involved in are only used in return
    1922                 :            :    statements.  ANALYZED is a bitmap that tracks which SSA names we have
    1923                 :            :    already started investigating.  */
    1924                 :            : 
    1925                 :            : static bool
    1926                 :    1361410 : ssa_name_only_returned_p (tree name, bitmap analyzed)
    1927                 :            : {
    1928                 :    1361410 :   bool res = true;
    1929                 :    1361410 :   imm_use_iterator imm_iter;
    1930                 :    1361410 :   gimple *stmt;
    1931                 :            : 
    1932                 :    1657070 :   FOR_EACH_IMM_USE_STMT (stmt, imm_iter, name)
    1933                 :            :     {
    1934                 :    1489240 :       if (is_gimple_debug (stmt))
    1935                 :      99453 :         continue;
    1936                 :            : 
    1937                 :    1389790 :       if (gimple_code (stmt) == GIMPLE_RETURN)
    1938                 :            :         {
    1939                 :      90986 :           tree t = gimple_return_retval (as_a <greturn *> (stmt));
    1940                 :      90986 :           if (t != name)
    1941                 :            :             {
    1942                 :          0 :               res = false;
    1943                 :          0 :               BREAK_FROM_IMM_USE_STMT (imm_iter);
    1944                 :            :             }
    1945                 :            :         }
    1946                 :    1298800 :       else if ((is_gimple_assign (stmt) && !gimple_has_volatile_ops (stmt))
    1947                 :    1305920 :                || gimple_code (stmt) == GIMPLE_PHI)
    1948                 :            :         {
    1949                 :            :           /* TODO: And perhaps for const function calls too?  */
    1950                 :     779133 :           tree lhs;
    1951                 :     779133 :           if (gimple_code (stmt) == GIMPLE_PHI)
    1952                 :     159027 :             lhs = gimple_phi_result (stmt);
    1953                 :            :           else
    1954                 :     620106 :             lhs = gimple_assign_lhs (stmt);
    1955                 :            : 
    1956                 :     779133 :           if (TREE_CODE (lhs) != SSA_NAME)
    1957                 :            :             {
    1958                 :     281165 :               res = false;
    1959                 :     281165 :               BREAK_FROM_IMM_USE_STMT (imm_iter);
    1960                 :            :             }
    1961                 :     836909 :           gcc_assert (!gimple_vdef (stmt));
    1962                 :     497968 :           if (bitmap_set_bit (analyzed, SSA_NAME_VERSION (lhs))
    1963                 :     497968 :               && !ssa_name_only_returned_p (lhs, analyzed))
    1964                 :            :             {
    1965                 :     392746 :               res = false;
    1966                 :     392746 :               BREAK_FROM_IMM_USE_STMT (imm_iter);
    1967                 :            :             }
    1968                 :            :         }
    1969                 :            :       else
    1970                 :            :         {
    1971                 :     519671 :           res = false;
    1972                 :     815332 :           BREAK_FROM_IMM_USE_STMT (imm_iter);
    1973                 :            :         }
    1974                 :            :     }
    1975                 :    1361410 :   return res;
    1976                 :            : }
    1977                 :            : 
    1978                 :            : /* Inspect the uses of the return value of the call associated with CS, and if
    1979                 :            :    it is not used or if it is only used to construct the return value of the
    1980                 :            :    caller, mark it as such in call or caller summary.  Also check for
    1981                 :            :    misaligned arguments.  */
    1982                 :            : 
    1983                 :            : static void
    1984                 :    2777280 : isra_analyze_call (cgraph_edge *cs)
    1985                 :            : {
    1986                 :    2777280 :   gcall *call_stmt = cs->call_stmt;
    1987                 :    2777280 :   unsigned count = gimple_call_num_args (call_stmt);
    1988                 :    2777280 :   isra_call_summary *csum = call_sums->get_create (cs);
    1989                 :            : 
    1990                 :    7855500 :   for (unsigned i = 0; i < count; i++)
    1991                 :            :     {
    1992                 :    5078220 :       tree arg = gimple_call_arg (call_stmt, i);
    1993                 :    5078220 :       if (is_gimple_reg (arg))
    1994                 :    2169210 :         continue;
    1995                 :            : 
    1996                 :    2909010 :       tree offset;
    1997                 :    2909010 :       poly_int64 bitsize, bitpos;
    1998                 :    2909010 :       machine_mode mode;
    1999                 :    2909010 :       int unsignedp, reversep, volatilep = 0;
    2000                 :    2909010 :       get_inner_reference (arg, &bitsize, &bitpos, &offset, &mode,
    2001                 :            :                            &unsignedp, &reversep, &volatilep);
    2002                 :    5818020 :       if (!multiple_p (bitpos, BITS_PER_UNIT))
    2003                 :            :         {
    2004                 :          0 :           csum->m_bit_aligned_arg = true;
    2005                 :          0 :           break;
    2006                 :            :         }
    2007                 :            :     }
    2008                 :            : 
    2009                 :    2777280 :   tree lhs = gimple_call_lhs (call_stmt);
    2010                 :    2777280 :   if (lhs)
    2011                 :            :     {
    2012                 :            :       /* TODO: Also detect aggregates on a LHS of a call that are only returned
    2013                 :            :          from this function (without being read anywhere).  */
    2014                 :    1139520 :       if (TREE_CODE (lhs) == SSA_NAME)
    2015                 :            :         {
    2016                 :     879128 :           bitmap analyzed = BITMAP_ALLOC (NULL);
    2017                 :     879128 :           if (ssa_name_only_returned_p (lhs, analyzed))
    2018                 :      78292 :             csum->m_return_returned = true;
    2019                 :     879128 :           BITMAP_FREE (analyzed);
    2020                 :            :         }
    2021                 :            :     }
    2022                 :            :   else
    2023                 :    1637750 :     csum->m_return_ignored = true;
    2024                 :    2777280 : }
    2025                 :            : 
    2026                 :            : /* Look at all calls going out of NODE, described also by IFS and perform all
    2027                 :            :    analyses necessary for IPA-SRA that are not done at body scan time or done
    2028                 :            :    even when body is not scanned because the function is not a candidate.  */
    2029                 :            : 
    2030                 :            : static void
    2031                 :     608127 : isra_analyze_all_outgoing_calls (cgraph_node *node)
    2032                 :            : {
    2033                 :    3293410 :   for (cgraph_edge *cs = node->callees; cs; cs = cs->next_callee)
    2034                 :    2685280 :     isra_analyze_call (cs);
    2035                 :     700123 :   for (cgraph_edge *cs = node->indirect_calls; cs; cs = cs->next_callee)
    2036                 :      91996 :     isra_analyze_call (cs);
    2037                 :     608127 : }
    2038                 :            : 
    2039                 :            : /* Dump a dereferences table with heading STR to file F.  */
    2040                 :            : 
    2041                 :            : static void
    2042                 :          0 : dump_dereferences_table (FILE *f, struct function *fun, const char *str)
    2043                 :            : {
    2044                 :          0 :   basic_block bb;
    2045                 :            : 
    2046                 :          0 :   fprintf (dump_file, "%s", str);
    2047                 :          0 :   FOR_BB_BETWEEN (bb, ENTRY_BLOCK_PTR_FOR_FN (fun),
    2048                 :            :                   EXIT_BLOCK_PTR_FOR_FN (fun), next_bb)
    2049                 :            :     {
    2050                 :          0 :       fprintf (f, "%4i  %i   ", bb->index, bitmap_bit_p (final_bbs, bb->index));
    2051                 :          0 :       if (bb != EXIT_BLOCK_PTR_FOR_FN (fun))
    2052                 :            :         {
    2053                 :            :           int i;
    2054                 :          0 :           for (i = 0; i < by_ref_count; i++)
    2055                 :            :             {
    2056                 :          0 :               int idx = bb->index * by_ref_count + i;
    2057                 :          0 :               fprintf (f, " %4" HOST_WIDE_INT_PRINT "d", bb_dereferences[idx]);
    2058                 :            :             }
    2059                 :            :         }
    2060                 :          0 :       fprintf (f, "\n");
    2061                 :            :     }
    2062                 :          0 :   fprintf (dump_file, "\n");
    2063                 :          0 : }
    2064                 :            : 
    2065                 :            : /* Propagate distances in bb_dereferences in the opposite direction than the
    2066                 :            :    control flow edges, in each step storing the maximum of the current value
    2067                 :            :    and the minimum of all successors.  These steps are repeated until the table
    2068                 :            :    stabilizes.  Note that BBs which might terminate the functions (according to
    2069                 :            :    final_bbs bitmap) never updated in this way.  */
    2070                 :            : 
    2071                 :            : static void
    2072                 :      52900 : propagate_dereference_distances (struct function *fun)
    2073                 :            : {
    2074                 :      52900 :   basic_block bb;
    2075                 :            : 
    2076                 :      52900 :   if (dump_file && (dump_flags & TDF_DETAILS))
    2077                 :          0 :     dump_dereferences_table (dump_file, fun,
    2078                 :            :                              "Dereference table before propagation:\n");
    2079                 :            : 
    2080                 :      52900 :   auto_vec<basic_block> queue (last_basic_block_for_fn (fun));
    2081                 :      52900 :   queue.quick_push (ENTRY_BLOCK_PTR_FOR_FN (fun));
    2082                 :     346580 :   FOR_EACH_BB_FN (bb, fun)
    2083                 :            :     {
    2084                 :     293680 :       queue.quick_push (bb);
    2085                 :     293680 :       bb->aux = bb;
    2086                 :            :     }
    2087                 :            : 
    2088                 :     402999 :   while (!queue.is_empty ())
    2089                 :            :     {
    2090                 :     350099 :       edge_iterator ei;
    2091                 :     350099 :       edge e;
    2092                 :     350099 :       bool change = false;
    2093                 :     350099 :       int i;
    2094                 :            : 
    2095                 :     350099 :       bb = queue.pop ();
    2096                 :     350099 :       bb->aux = NULL;
    2097                 :            : 
    2098                 :     350099 :       if (bitmap_bit_p (final_bbs, bb->index))
    2099                 :     112381 :         continue;
    2100                 :            : 
    2101                 :    9767300 :       for (i = 0; i < by_ref_count; i++)
    2102                 :            :         {
    2103                 :    9529590 :           int idx = bb->index * by_ref_count + i;
    2104                 :    9529590 :           bool first = true;
    2105                 :    9529590 :           HOST_WIDE_INT inh = 0;
    2106                 :            : 
    2107                 :   22510400 :           FOR_EACH_EDGE (e, ei, bb->succs)
    2108                 :            :           {
    2109                 :   12980800 :             int succ_idx = e->dest->index * by_ref_count + i;
    2110                 :            : 
    2111                 :   12980800 :             if (e->dest == EXIT_BLOCK_PTR_FOR_FN (fun))
    2112                 :          0 :               continue;
    2113                 :            : 
    2114                 :   12980800 :             if (first)
    2115                 :            :               {
    2116                 :    9526860 :                 first = false;
    2117                 :    9526860 :                 inh = bb_dereferences [succ_idx];
    2118                 :            :               }
    2119                 :    3453940 :             else if (bb_dereferences [succ_idx] < inh)
    2120                 :            :               inh = bb_dereferences [succ_idx];
    2121                 :            :           }
    2122                 :            : 
    2123                 :    9529590 :           if (!first && bb_dereferences[idx] < inh)
    2124                 :            :             {
    2125                 :      46670 :               bb_dereferences[idx] = inh;
    2126                 :      46670 :               change = true;
    2127                 :            :             }
    2128                 :            :         }
    2129                 :            : 
    2130                 :     237718 :       if (change)
    2131                 :      49129 :         FOR_EACH_EDGE (e, ei, bb->preds)
    2132                 :            :           {
    2133                 :       8031 :             if (e->src->aux)
    2134                 :       4512 :               continue;
    2135                 :            : 
    2136                 :       3519 :             e->src->aux = e->src;
    2137                 :      11550 :             queue.quick_push (e->src);
    2138                 :            :           }
    2139                 :            :     }
    2140                 :            : 
    2141                 :      52900 :   if (dump_file && (dump_flags & TDF_DETAILS))
    2142                 :          0 :     dump_dereferences_table (dump_file, fun,
    2143                 :            :                              "Dereference table after propagation:\n");
    2144                 :      52900 : }
    2145                 :            : 
    2146                 :            : /* Perform basic checks on ACCESS to PARM described by DESC and all its
    2147                 :            :    children, return true if the parameter cannot be split, otherwise return
    2148                 :            :    true and update *TOTAL_SIZE and *ONLY_CALLS.  ENTRY_BB_INDEX must be the
    2149                 :            :    index of the entry BB in the function of PARM.  */
    2150                 :            : 
    2151                 :            : static bool
    2152                 :     221259 : check_gensum_access (tree parm, gensum_param_desc *desc,
    2153                 :            :                      gensum_param_access *access,
    2154                 :            :                      HOST_WIDE_INT *nonarg_acc_size, bool *only_calls,
    2155                 :            :                       int entry_bb_index)
    2156                 :            : {
    2157                 :     221259 :   if (access->nonarg)
    2158                 :            :     {
    2159                 :     184490 :       *only_calls = false;
    2160                 :     184490 :       *nonarg_acc_size += access->size;
    2161                 :            : 
    2162                 :     184490 :       if (access->first_child)
    2163                 :            :         {
    2164                 :        326 :           disqualify_split_candidate (desc, "Overlapping non-call uses.");
    2165                 :        326 :           return true;
    2166                 :            :         }
    2167                 :            :     }
    2168                 :            :   /* Do not decompose a non-BLKmode param in a way that would create
    2169                 :            :      BLKmode params.  Especially for by-reference passing (thus,
    2170                 :            :      pointer-type param) this is hardly worthwhile.  */
    2171                 :     220933 :   if (DECL_MODE (parm) != BLKmode
    2172                 :     220933 :       && TYPE_MODE (access->type) == BLKmode)
    2173                 :            :     {
    2174                 :       1157 :       disqualify_split_candidate (desc, "Would convert a non-BLK to a BLK.");
    2175                 :       1157 :       return true;
    2176                 :            :     }
    2177                 :            : 
    2178                 :     219776 :   if (desc->by_ref)
    2179                 :            :     {
    2180                 :      87407 :       int idx = (entry_bb_index * by_ref_count + desc->deref_index);
    2181                 :      87407 :       if ((access->offset + access->size) > bb_dereferences[idx])
    2182                 :            :         {
    2183                 :      21041 :           disqualify_split_candidate (desc, "Would create a possibly "
    2184                 :            :                                       "illegal dereference in a caller.");
    2185                 :      21041 :           return true;
    2186                 :            :         }
    2187                 :            :     }
    2188                 :            : 
    2189                 :     198735 :   for (gensum_param_access *ch = access->first_child;
    2190                 :     215248 :        ch;
    2191                 :      16513 :        ch = ch->next_sibling)
    2192                 :      16513 :     if (check_gensum_access (parm, desc, ch, nonarg_acc_size, only_calls,
    2193                 :            :                              entry_bb_index))
    2194                 :            :       return true;
    2195                 :            : 
    2196                 :            :   return false;
    2197                 :            : }
    2198                 :            : 
    2199                 :            : /* Copy data from FROM and all of its children to a vector of accesses in IPA
    2200                 :            :    descriptor DESC.  */
    2201                 :            : 
    2202                 :            : static void
    2203                 :     255923 : copy_accesses_to_ipa_desc (gensum_param_access *from, isra_param_desc *desc)
    2204                 :            : {
    2205                 :     255923 :   param_access *to = ggc_cleared_alloc<param_access> ();
    2206                 :     255923 :   gcc_checking_assert ((from->offset % BITS_PER_UNIT) == 0);
    2207                 :     255923 :   gcc_checking_assert ((from->size % BITS_PER_UNIT) == 0);
    2208                 :     255923 :   to->unit_offset = from->offset / BITS_PER_UNIT;
    2209                 :     255923 :   to->unit_size = from->size / BITS_PER_UNIT;
    2210                 :     255923 :   to->type = from->type;
    2211                 :     255923 :   to->alias_ptr_type = from->alias_ptr_type;
    2212                 :     255923 :   to->certain = from->nonarg;
    2213                 :     255923 :   to->reverse = from->reverse;
    2214                 :     255923 :   vec_safe_push (desc->accesses, to);
    2215                 :            : 
    2216                 :     255923 :   for (gensum_param_access *ch = from->first_child;
    2217                 :     274255 :        ch;
    2218                 :      18332 :        ch = ch->next_sibling)
    2219                 :      18332 :     copy_accesses_to_ipa_desc (ch, desc);
    2220                 :     255923 : }
    2221                 :            : 
    2222                 :            : /* Analyze function body scan results stored in param_accesses and
    2223                 :            :    param_accesses, detect possible transformations and store information of
    2224                 :            :    those in function summary.  NODE, FUN and IFS are all various structures
    2225                 :            :    describing the currently analyzed function.  */
    2226                 :            : 
    2227                 :            : static void
    2228                 :     492542 : process_scan_results (cgraph_node *node, struct function *fun,
    2229                 :            :                       isra_func_summary *ifs,
    2230                 :            :                       vec<gensum_param_desc> *param_descriptions)
    2231                 :            : {
    2232                 :     492542 :   bool check_pass_throughs = false;
    2233                 :     492542 :   bool dereferences_propagated = false;
    2234                 :     492542 :   tree parm = DECL_ARGUMENTS (node->decl);
    2235                 :     492542 :   unsigned param_count = param_descriptions->length();
    2236                 :            : 
    2237                 :    2012600 :   for (unsigned desc_index = 0;
    2238                 :    2012600 :        desc_index < param_count;
    2239                 :    1520060 :        desc_index++, parm = DECL_CHAIN (parm))
    2240                 :            :     {
    2241                 :    1520060 :       gensum_param_desc *desc = &(*param_descriptions)[desc_index];
    2242                 :    1520060 :       if (!desc->split_candidate)
    2243                 :    1239730 :         continue;
    2244                 :            : 
    2245                 :     302856 :       if (flag_checking)
    2246                 :     302856 :         isra_verify_access_tree (desc->accesses);
    2247                 :            : 
    2248                 :     302856 :       if (!dereferences_propagated
    2249                 :     285704 :           && desc->by_ref
    2250                 :     186118 :           && desc->accesses)
    2251                 :            :         {
    2252                 :      52900 :           propagate_dereference_distances (fun);
    2253                 :      52900 :           dereferences_propagated = true;
    2254                 :            :         }
    2255                 :            : 
    2256                 :     302856 :       HOST_WIDE_INT nonarg_acc_size = 0;
    2257                 :     302856 :       bool only_calls = true;
    2258                 :     302856 :       bool check_failed = false;
    2259                 :            : 
    2260                 :     302856 :       int entry_bb_index = ENTRY_BLOCK_PTR_FOR_FN (fun)->index;
    2261                 :     302856 :       for (gensum_param_access *acc = desc->accesses;
    2262                 :     485078 :            acc;
    2263                 :     182222 :            acc = acc->next_sibling)
    2264                 :     204746 :         if (check_gensum_access (parm, desc, acc, &nonarg_acc_size, &only_calls,
    2265                 :            :                                  entry_bb_index))
    2266                 :            :           {
    2267                 :            :             check_failed = true;
    2268                 :            :             break;
    2269                 :            :           }
    2270                 :     302856 :       if (check_failed)
    2271                 :      22524 :         continue;
    2272                 :            : 
    2273                 :     280332 :       if (only_calls)
    2274                 :     184746 :         desc->locally_unused = true;
    2275                 :            : 
    2276                 :     280332 :       HOST_WIDE_INT cur_param_size
    2277                 :     280332 :         = tree_to_uhwi (TYPE_SIZE (TREE_TYPE (parm)));
    2278                 :     280332 :       HOST_WIDE_INT param_size_limit;
    2279                 :     280332 :       if (!desc->by_ref || optimize_function_for_size_p (fun))
    2280                 :            :         param_size_limit = cur_param_size;
    2281                 :            :       else
    2282                 :     168206 :           param_size_limit
    2283                 :     168206 :             = opt_for_fn (node->decl,
    2284                 :            :                           param_ipa_sra_ptr_growth_factor) * cur_param_size;
    2285                 :     280332 :       if (nonarg_acc_size > param_size_limit
    2286                 :     274581 :           || (!desc->by_ref && nonarg_acc_size == param_size_limit))
    2287                 :            :         {
    2288                 :      52162 :           disqualify_split_candidate (desc, "Would result into a too big set "
    2289                 :            :                                       "of replacements.");
    2290                 :            :         }
    2291                 :            :       else
    2292                 :            :         {
    2293                 :            :           /* create_parameter_descriptors makes sure unit sizes of all
    2294                 :            :              candidate parameters fit unsigned integers restricted to
    2295                 :            :              ISRA_ARG_SIZE_LIMIT.  */
    2296                 :     228170 :           desc->param_size_limit = param_size_limit / BITS_PER_UNIT;
    2297                 :     228170 :           desc->nonarg_acc_size = nonarg_acc_size / BITS_PER_UNIT;
    2298                 :     228170 :           if (desc->split_candidate && desc->ptr_pt_count)
    2299                 :            :             {
    2300                 :     110173 :               gcc_assert (desc->by_ref);
    2301                 :            :               check_pass_throughs = true;
    2302                 :            :             }
    2303                 :            :         }
    2304                 :            :     }
    2305                 :            : 
    2306                 :            :   /* When a pointer parameter is passed-through to a callee, in which it is
    2307                 :            :      only used to read only one or a few items, we can attempt to transform it
    2308                 :            :      to obtaining and passing through the items instead of the pointer.  But we
    2309                 :            :      must take extra care that 1) we do not introduce any segfault by moving
    2310                 :            :      dereferences above control flow and that 2) the data is not modified
    2311                 :            :      through an alias in this function.  The IPA analysis must not introduce
    2312                 :            :      any accesses candidates unless it can prove both.
    2313                 :            : 
    2314                 :            :      The current solution is very crude as it consists of ensuring that the
    2315                 :            :      call postdominates entry BB and that the definition of VUSE of the call is
    2316                 :            :      default definition.  TODO: For non-recursive callees in the same
    2317                 :            :      compilation unit we could do better by doing analysis in topological order
    2318                 :            :      an looking into access candidates of callees, using their alias_ptr_types
    2319                 :            :      to attempt real AA.  We could also use the maximum known dereferenced
    2320                 :            :      offset in this function at IPA level.
    2321                 :            : 
    2322                 :            :      TODO: Measure the overhead and the effect of just being pessimistic.
    2323                 :            :      Maybe this is only -O3 material?
    2324                 :            :   */
    2325                 :     492542 :   bool pdoms_calculated = false;
    2326                 :     492542 :   if (check_pass_throughs)
    2327                 :     600716 :     for (cgraph_edge *cs = node->callees; cs; cs = cs->next_callee)
    2328                 :            :       {
    2329                 :     511261 :         gcall *call_stmt = cs->call_stmt;
    2330                 :     511261 :         tree vuse = gimple_vuse (call_stmt);
    2331                 :            : 
    2332                 :            :         /* If the callee is a const function, we don't get a VUSE.  In such
    2333                 :            :            case there will be no memory accesses in the called function (or the
    2334                 :            :            const attribute is wrong) and then we just don't care.  */
    2335                 :    1017120 :         bool uses_memory_as_obtained = vuse && SSA_NAME_IS_DEFAULT_DEF (vuse);
    2336                 :            : 
    2337                 :     511261 :         unsigned count = gimple_call_num_args (call_stmt);
    2338                 :     511261 :         isra_call_summary *csum = call_sums->get_create (cs);
    2339                 :     511261 :         csum->init_inputs (count);
    2340                 :    1540180 :         for (unsigned argidx = 0; argidx < count; argidx++)
    2341                 :            :           {
    2342                 :    1028920 :             if (!csum->m_arg_flow[argidx].pointer_pass_through)
    2343                 :     850631 :               continue;
    2344                 :     178287 :             unsigned pidx
    2345                 :     178287 :               = get_single_param_flow_source (&csum->m_arg_flow[argidx]);
    2346                 :     178287 :             gensum_param_desc *desc = &(*param_descriptions)[pidx];
    2347                 :     178287 :             if (!desc->split_candidate)
    2348                 :            :               {
    2349                 :      17545 :                 csum->m_arg_flow[argidx].pointer_pass_through = false;
    2350                 :      17545 :                 continue;
    2351                 :            :               }
    2352                 :     160742 :             if (!uses_memory_as_obtained)
    2353                 :      96546 :               continue;
    2354                 :            : 
    2355                 :            :             /* Post-dominator check placed last, hoping that it usually won't
    2356                 :            :                be needed.  */
    2357                 :      64196 :             if (!pdoms_calculated)
    2358                 :            :               {
    2359                 :      50700 :                 gcc_checking_assert (cfun);
    2360                 :      50700 :                 add_noreturn_fake_exit_edges ();
    2361                 :      50700 :                 connect_infinite_loops_to_exit ();
    2362                 :      50700 :                 calculate_dominance_info (CDI_POST_DOMINATORS);
    2363                 :      50700 :                 pdoms_calculated = true;
    2364                 :            :               }
    2365                 :      64196 :             if (dominated_by_p (CDI_POST_DOMINATORS,
    2366                 :      64196 :                                 gimple_bb (call_stmt),
    2367                 :      64196 :                                 single_succ (ENTRY_BLOCK_PTR_FOR_FN (fun))))
    2368                 :      43600 :               csum->m_arg_flow[argidx].safe_to_import_accesses = true;
    2369                 :            :           }
    2370                 :            : 
    2371                 :            :       }
    2372                 :      89455 :   if (pdoms_calculated)
    2373                 :            :     {
    2374                 :      50700 :       free_dominance_info (CDI_POST_DOMINATORS);
    2375                 :      50700 :       remove_fake_exit_edges ();
    2376                 :            :     }
    2377                 :            : 
    2378                 :            :   /* TODO: Add early exit if we disqualified everything.  This also requires
    2379                 :            :      that we either relax the restriction that
    2380                 :            :      ipa_param_adjustments.m_always_copy_start must be the number of PARM_DECLs
    2381                 :            :      or store the number of parameters to IPA-SRA function summary and use that
    2382                 :            :      when just removing params.  */
    2383                 :            : 
    2384                 :     492542 :   vec_safe_reserve_exact (ifs->m_parameters, param_count);
    2385                 :     492542 :   ifs->m_parameters->quick_grow_cleared (param_count);
    2386                 :    2012600 :   for (unsigned desc_index = 0; desc_index < param_count; desc_index++)
    2387                 :            :     {
    2388                 :    1520060 :       gensum_param_desc *s = &(*param_descriptions)[desc_index];
    2389                 :    1520060 :       isra_param_desc *d = &(*ifs->m_parameters)[desc_index];
    2390                 :            : 
    2391                 :    1520060 :       d->param_size_limit = s->param_size_limit;
    2392                 :    1520060 :       d->size_reached = s->nonarg_acc_size;
    2393                 :    1520060 :       d->locally_unused = s->locally_unused;
    2394                 :    1520060 :       d->split_candidate = s->split_candidate;
    2395                 :    1520060 :       d->by_ref = s->by_ref;
    2396                 :            : 
    2397                 :    1520060 :       for (gensum_param_access *acc = s->accesses;
    2398                 :    1757650 :            acc;
    2399                 :     237591 :            acc = acc->next_sibling)
    2400                 :     237591 :         copy_accesses_to_ipa_desc (acc, d);
    2401                 :            :     }
    2402                 :            : 
    2403                 :     492542 :   if (dump_file)
    2404                 :         78 :     dump_isra_param_descriptors (dump_file, node->decl, ifs);
    2405                 :     492542 : }
    2406                 :            : 
    2407                 :            : /* Return true if there are any overlaps among certain accesses of DESC.  If
    2408                 :            :    non-NULL, set *CERTAIN_ACCESS_PRESENT_P upon encountering a certain access
    2409                 :            :    too.  DESC is assumed to be a split candidate that is not locally
    2410                 :            :    unused.  */
    2411                 :            : 
    2412                 :            : static bool
    2413                 :      59565 : overlapping_certain_accesses_p (isra_param_desc *desc,
    2414                 :            :                                 bool *certain_access_present_p)
    2415                 :            : {
    2416                 :      59565 :   unsigned pclen = vec_safe_length (desc->accesses);
    2417                 :     154052 :   for (unsigned i = 0; i < pclen; i++)
    2418                 :            :     {
    2419                 :      95390 :       param_access *a1 = (*desc->accesses)[i];
    2420                 :            : 
    2421                 :      95390 :       if (!a1->certain)
    2422                 :       4203 :         continue;
    2423                 :      91187 :       if (certain_access_present_p)
    2424                 :      83876 :         *certain_access_present_p = true;
    2425                 :     130326 :       for (unsigned j = i + 1; j < pclen; j++)
    2426                 :            :         {
    2427                 :      40042 :           param_access *a2 = (*desc->accesses)[j];
    2428                 :      40042 :           if (a2->certain
    2429                 :      40042 :               && a1->unit_offset < a2->unit_offset + a2->unit_size
    2430                 :      39816 :               && a1->unit_offset + a1->unit_size > a2->unit_offset)
    2431                 :            :             return true;
    2432                 :            :         }
    2433                 :            :     }
    2434                 :            :   return false;
    2435                 :            : }
    2436                 :            : 
    2437                 :            : /* Check for any overlaps of certain param accesses among splitting candidates
    2438                 :            :    and signal an ICE if there are any.  If CERTAIN_MUST_EXIST is set, also
    2439                 :            :    check that used splitting candidates have at least one certain access.  */
    2440                 :            : 
    2441                 :            : static void
    2442                 :    1478760 : verify_splitting_accesses (cgraph_node *node, bool certain_must_exist)
    2443                 :            : {
    2444                 :    1478760 :   isra_func_summary *ifs = func_sums->get (node);
    2445                 :    1478760 :   if (!ifs || !ifs->m_candidate)
    2446                 :     671293 :     return;
    2447                 :     807471 :   unsigned param_count = vec_safe_length (ifs->m_parameters);
    2448                 :    2731610 :   for (unsigned pidx = 0; pidx < param_count; pidx++)
    2449                 :            :     {
    2450                 :    1924140 :       isra_param_desc *desc = &(*ifs->m_parameters)[pidx];
    2451                 :    1924140 :       if (!desc->split_candidate || desc->locally_unused)
    2452                 :    1871830 :         continue;
    2453                 :            : 
    2454                 :      52307 :       bool certain_access_present = !certain_must_exist;
    2455                 :      52307 :       if (overlapping_certain_accesses_p (desc, &certain_access_present))
    2456                 :          0 :         internal_error ("Function %qs, parameter %u, has IPA-SRA accesses "
    2457                 :            :                         "which overlap", node->dump_name (), pidx);
    2458                 :      52307 :       if (!certain_access_present)
    2459                 :          0 :         internal_error ("Function %s, parameter %u, is used but does not "
    2460                 :            :                         "have any certain IPA-SRA access",
    2461                 :            :                         node->dump_name (), pidx);
    2462                 :            :     }
    2463                 :            : }
    2464                 :            : 
    2465                 :            : /* Intraprocedural part of IPA-SRA analysis.  Scan function body of NODE and
    2466                 :            :    create a summary structure describing IPA-SRA opportunities and constraints
    2467                 :            :    in it.  */
    2468                 :            : 
    2469                 :            : static void
    2470                 :     807382 : ipa_sra_summarize_function (cgraph_node *node)
    2471                 :            : {
    2472                 :     807382 :   if (dump_file)
    2473                 :        114 :     fprintf (dump_file, "Creating summary for %s/%i:\n", node->name (),
    2474                 :            :              node->order);
    2475                 :     807382 :   if (!ipa_sra_preliminary_function_checks (node))
    2476                 :            :     return;
    2477                 :     608127 :   gcc_obstack_init (&gensum_obstack);
    2478                 :     608127 :   isra_func_summary *ifs = func_sums->get_create (node);
    2479                 :     608127 :   ifs->m_candidate = true;
    2480                 :     608127 :   tree ret = TREE_TYPE (TREE_TYPE (node->decl));
    2481                 :     608127 :   ifs->m_returns_value = (TREE_CODE (ret) != VOID_TYPE);
    2482                 :            : 
    2483                 :     608127 :   decl2desc = new hash_map<tree, gensum_param_desc *>;
    2484                 :     608127 :   unsigned count = 0;
    2485                 :    2128190 :   for (tree parm = DECL_ARGUMENTS (node->decl); parm; parm = DECL_CHAIN (parm))
    2486                 :    1520060 :     count++;
    2487                 :            : 
    2488                 :     608127 :   if (count > 0)
    2489                 :            :     {
    2490                 :     985084 :       auto_vec<gensum_param_desc, 16> param_descriptions (count);
    2491                 :     492542 :       param_descriptions.reserve_exact (count);
    2492                 :     492542 :       param_descriptions.quick_grow_cleared (count);
    2493                 :            : 
    2494                 :     492542 :       bool cfun_pushed = false;
    2495                 :     492542 :       struct function *fun = DECL_STRUCT_FUNCTION (node->decl);
    2496                 :     492542 :       if (create_parameter_descriptors (node, &param_descriptions))
    2497                 :            :         {
    2498                 :     244588 :           push_cfun (fun);
    2499                 :     244588 :           cfun_pushed = true;
    2500                 :     244588 :           final_bbs = BITMAP_ALLOC (NULL);
    2501                 :     244588 :           bb_dereferences = XCNEWVEC (HOST_WIDE_INT,
    2502                 :            :                                       by_ref_count
    2503                 :            :                                       * last_basic_block_for_fn (fun));
    2504                 :     244588 :           aa_walking_limit = opt_for_fn (node->decl, param_ipa_max_aa_steps);
    2505                 :     244588 :           scan_function (node, fun);
    2506                 :            : 
    2507                 :     244588 :           if (dump_file)
    2508                 :            :             {
    2509                 :         43 :               dump_gensum_param_descriptors (dump_file, node->decl,
    2510                 :            :                                              &param_descriptions);
    2511                 :         43 :               fprintf (dump_file, "----------------------------------------\n");
    2512                 :            :             }
    2513                 :            :         }
    2514                 :     492542 :       process_scan_results (node, fun, ifs, &param_descriptions);
    2515                 :            : 
    2516                 :     492542 :       if (cfun_pushed)
    2517                 :     244588 :         pop_cfun ();
    2518                 :     492542 :       if (bb_dereferences)
    2519                 :            :         {
    2520                 :     244588 :           free (bb_dereferences);
    2521                 :     244588 :           bb_dereferences = NULL;
    2522                 :     244588 :           BITMAP_FREE (final_bbs);
    2523                 :     244588 :           final_bbs = NULL;
    2524                 :            :         }
    2525                 :            :     }
    2526                 :     608127 :   isra_analyze_all_outgoing_calls (node);
    2527                 :            : 
    2528                 :    1216250 :   delete decl2desc;
    2529                 :     608127 :   decl2desc = NULL;
    2530                 :     608127 :   obstack_free (&gensum_obstack, NULL);
    2531                 :     608127 :   if (dump_file)
    2532                 :        109 :     fprintf (dump_file, "\n\n");
    2533                 :     608127 :   if (flag_checking)
    2534                 :     608121 :     verify_splitting_accesses (node, false);
    2535                 :            :   return;
    2536                 :            : }
    2537                 :            : 
    2538                 :            : /* Intraprocedural part of IPA-SRA analysis.  Scan bodies of all functions in
    2539                 :            :    this compilation unit and create summary structures describing IPA-SRA
    2540                 :            :    opportunities and constraints in them.  */
    2541                 :            : 
    2542                 :            : static void
    2543                 :      92615 : ipa_sra_generate_summary (void)
    2544                 :            : {
    2545                 :      92615 :   struct cgraph_node *node;
    2546                 :            : 
    2547                 :      92615 :   gcc_checking_assert (!func_sums);
    2548                 :      92615 :   gcc_checking_assert (!call_sums);
    2549                 :      92615 :   func_sums
    2550                 :      92615 :     = (new (ggc_alloc_no_dtor <ipa_sra_function_summaries> ())
    2551                 :      92615 :        ipa_sra_function_summaries (symtab, true));
    2552                 :      92615 :   call_sums = new ipa_sra_call_summaries (symtab);
    2553                 :            : 
    2554                 :    1799990 :   FOR_EACH_FUNCTION_WITH_GIMPLE_BODY (node)
    2555                 :     807382 :     ipa_sra_summarize_function (node);
    2556                 :      92615 :   return;
    2557                 :            : }
    2558                 :            : 
    2559                 :            : /* Write intraprocedural analysis information about E and all of its outgoing
    2560                 :            :    edges into a stream for LTO WPA.  */
    2561                 :            : 
    2562                 :            : static void
    2563                 :     133269 : isra_write_edge_summary (output_block *ob, cgraph_edge *e)
    2564                 :            : {
    2565                 :     133269 :   isra_call_summary *csum = call_sums->get (e);
    2566                 :     133269 :   unsigned input_count = csum->m_arg_flow.length ();
    2567                 :     133269 :   streamer_write_uhwi (ob, input_count);
    2568                 :     148074 :   for (unsigned i = 0; i < input_count; i++)
    2569                 :            :     {
    2570                 :      14805 :       isra_param_flow *ipf = &csum->m_arg_flow[i];
    2571                 :      14805 :       streamer_write_hwi (ob, ipf->length);
    2572                 :      14805 :       bitpack_d bp = bitpack_create (ob->main_stream);
    2573                 :      24094 :       for (int j = 0; j < ipf->length; j++)
    2574                 :       9289 :         bp_pack_value (&bp, ipf->inputs[j], 8);
    2575                 :      14805 :       bp_pack_value (&bp, ipf->aggregate_pass_through, 1);
    2576                 :      14805 :       bp_pack_value (&bp, ipf->pointer_pass_through, 1);
    2577                 :      14805 :       bp_pack_value (&bp, ipf->safe_to_import_accesses, 1);
    2578                 :      14805 :       streamer_write_bitpack (&bp);
    2579                 :      14805 :       streamer_write_uhwi (ob, ipf->unit_offset);
    2580                 :      14805 :       streamer_write_uhwi (ob, ipf->unit_size);
    2581                 :            :     }
    2582                 :     133269 :   bitpack_d bp = bitpack_create (ob->main_stream);
    2583                 :     133269 :   bp_pack_value (&bp, csum->m_return_ignored, 1);
    2584                 :     133269 :   bp_pack_value (&bp, csum->m_return_returned, 1);
    2585                 :     133269 :   bp_pack_value (&bp, csum->m_bit_aligned_arg, 1);
    2586                 :     133269 :   streamer_write_bitpack (&bp);
    2587                 :     133269 : }
    2588                 :            : 
    2589                 :            : /* Write intraprocedural analysis information about NODE and all of its outgoing
    2590                 :            :    edges into a stream for LTO WPA.  */
    2591                 :            : 
    2592                 :            : static void
    2593                 :      35174 : isra_write_node_summary (output_block *ob, cgraph_node *node)
    2594                 :            : {
    2595                 :      35174 :   isra_func_summary *ifs = func_sums->get (node);
    2596                 :      35174 :   lto_symtab_encoder_t encoder = ob->decl_state->symtab_node_encoder;
    2597                 :      35174 :   int node_ref = lto_symtab_encoder_encode (encoder, node);
    2598                 :      35174 :   streamer_write_uhwi (ob, node_ref);
    2599                 :            : 
    2600                 :      35174 :   unsigned param_desc_count = vec_safe_length (ifs->m_parameters);
    2601                 :      35174 :   streamer_write_uhwi (ob, param_desc_count);
    2602                 :     271980 :   for (unsigned i = 0; i < param_desc_count; i++)
    2603                 :            :     {
    2604                 :     236806 :       isra_param_desc *desc = &(*ifs->m_parameters)[i];
    2605                 :     236806 :       unsigned access_count = vec_safe_length (desc->accesses);
    2606                 :     236806 :       streamer_write_uhwi (ob, access_count);
    2607                 :     239168 :       for (unsigned j = 0; j < access_count; j++)
    2608                 :            :         {
    2609                 :       2362 :           param_access *acc = (*desc->accesses)[j];
    2610                 :       2362 :           stream_write_tree (ob, acc->type, true);
    2611                 :       2362 :           stream_write_tree (ob, acc->alias_ptr_type, true);
    2612                 :       2362 :           streamer_write_uhwi (ob, acc->unit_offset);
    2613                 :       2362 :           streamer_write_uhwi (ob, acc->unit_size);
    2614                 :       2362 :           bitpack_d bp = bitpack_create (ob->main_stream);
    2615                 :       2362 :           bp_pack_value (&bp, acc->certain, 1);
    2616                 :       2362 :           streamer_write_bitpack (&bp);
    2617                 :            :         }
    2618                 :     236806 :       streamer_write_uhwi (ob, desc->param_size_limit);
    2619                 :     236806 :       streamer_write_uhwi (ob, desc->size_reached);
    2620                 :     236806 :       bitpack_d bp = bitpack_create (ob->main_stream);
    2621                 :     236806 :       bp_pack_value (&bp, desc->locally_unused, 1);
    2622                 :     236806 :       bp_pack_value (&bp, desc->split_candidate, 1);
    2623                 :     236806 :       bp_pack_value (&bp, desc->by_ref, 1);
    2624                 :     236806 :       streamer_write_bitpack (&bp);
    2625                 :            :     }
    2626                 :      35174 :   bitpack_d bp = bitpack_create (ob->main_stream);
    2627                 :      35174 :   bp_pack_value (&bp, ifs->m_candidate, 1);
    2628                 :      35174 :   bp_pack_value (&bp, ifs->m_returns_value, 1);
    2629                 :      35174 :   bp_pack_value (&bp, ifs->m_return_ignored, 1);
    2630                 :      35174 :   gcc_assert (!ifs->m_queued);
    2631                 :      35174 :   streamer_write_bitpack (&bp);
    2632                 :            : 
    2633                 :     167099 :   for (cgraph_edge *e = node->callees; e; e = e->next_callee)
    2634                 :     131925 :     isra_write_edge_summary (ob, e);
    2635                 :      36518 :   for (cgraph_edge *e = node->indirect_calls; e; e = e->next_callee)
    2636                 :       1344 :     isra_write_edge_summary (ob, e);
    2637                 :      35174 : }
    2638                 :            : 
    2639                 :            : /* Write intraprocedural analysis information into a stream for LTO WPA.  */
    2640                 :            : 
    2641                 :            : static void
    2642                 :      15025 : ipa_sra_write_summary (void)
    2643                 :            : {
    2644                 :      15025 :   if (!func_sums || !call_sums)
    2645                 :          0 :     return;
    2646                 :            : 
    2647                 :      15025 :   struct output_block *ob = create_output_block (LTO_section_ipa_sra);
    2648                 :      15025 :   lto_symtab_encoder_t encoder = ob->decl_state->symtab_node_encoder;
    2649                 :      15025 :   ob->symbol = NULL;
    2650                 :            : 
    2651                 :      15025 :   unsigned int count = 0;
    2652                 :      15025 :   lto_symtab_encoder_iterator lsei;
    2653                 :      92867 :   for (lsei = lsei_start_function_in_partition (encoder);
    2654                 :     185566 :        !lsei_end_p (lsei);
    2655                 :      77842 :        lsei_next_function_in_partition (&lsei))
    2656                 :            :     {
    2657                 :      77842 :       cgraph_node *node = lsei_cgraph_node (lsei);
    2658                 :     155684 :       if (node->has_gimple_body_p ()
    2659                 :     153890 :           && func_sums->get (node) != NULL)
    2660                 :      35174 :         count++;
    2661                 :            :     }
    2662                 :      15025 :   streamer_write_uhwi (ob, count);
    2663                 :            : 
    2664                 :            :   /* Process all of the functions.  */
    2665                 :     185566 :   for (lsei = lsei_start_function_in_partition (encoder); !lsei_end_p (lsei);
    2666                 :      77842 :        lsei_next_function_in_partition (&lsei))
    2667                 :            :     {
    2668                 :      77842 :       cgraph_node *node = lsei_cgraph_node (lsei);
    2669                 :     155684 :       if (node->has_gimple_body_p ()
    2670                 :     153890 :           && func_sums->get (node) != NULL)
    2671                 :      35174 :         isra_write_node_summary (ob, node);
    2672                 :            :     }
    2673                 :      15025 :   streamer_write_char_stream (ob->main_stream, 0);
    2674                 :      15025 :   produce_asm (ob, NULL);
    2675                 :      15025 :   destroy_output_block (ob);
    2676                 :            : }
    2677                 :            : 
    2678                 :            : /* Read intraprocedural analysis information about E and all of its outgoing
    2679                 :            :    edges into a stream for LTO WPA.  */
    2680                 :            : 
    2681                 :            : static void
    2682                 :     108215 : isra_read_edge_summary (struct lto_input_block *ib, cgraph_edge *cs)
    2683                 :            : {
    2684                 :     108215 :   isra_call_summary *csum = call_sums->get_create (cs);
    2685                 :     108215 :   unsigned input_count = streamer_read_uhwi (ib);
    2686                 :     108215 :   csum->init_inputs (input_count);
    2687                 :     113584 :   for (unsigned i = 0; i < input_count; i++)
    2688                 :            :     {
    2689                 :       5369 :       isra_param_flow *ipf = &csum->m_arg_flow[i];
    2690                 :       5369 :       ipf->length = streamer_read_hwi (ib);
    2691                 :       5369 :       bitpack_d bp = streamer_read_bitpack (ib);
    2692                 :       8175 :       for (int j = 0; j < ipf->length; j++)
    2693                 :       2806 :         ipf->inputs[j] = bp_unpack_value (&bp, 8);
    2694                 :       5369 :       ipf->aggregate_pass_through = bp_unpack_value (&bp, 1);
    2695                 :       5369 :       ipf->pointer_pass_through = bp_unpack_value (&bp, 1);
    2696                 :       5369 :       ipf->safe_to_import_accesses = bp_unpack_value (&bp, 1);
    2697                 :       5369 :       ipf->unit_offset = streamer_read_uhwi (ib);
    2698                 :       5369 :       ipf->unit_size = streamer_read_uhwi (ib);
    2699                 :            :     }
    2700                 :     108215 :   bitpack_d bp = streamer_read_bitpack (ib);
    2701                 :     108215 :   csum->m_return_ignored = bp_unpack_value (&bp, 1);
    2702                 :     108215 :   csum->m_return_returned = bp_unpack_value (&bp, 1);
    2703                 :     108215 :   csum->m_bit_aligned_arg = bp_unpack_value (&bp, 1);
    2704                 :     108215 : }
    2705                 :            : 
    2706                 :            : /* Read intraprocedural analysis information about NODE and all of its outgoing
    2707                 :            :    edges into a stream for LTO WPA.  */
    2708                 :            : 
    2709                 :            : static void
    2710                 :      24360 : isra_read_node_info (struct lto_input_block *ib, cgraph_node *node,
    2711                 :            :                      struct data_in *data_in)
    2712                 :            : {
    2713                 :      24360 :   isra_func_summary *ifs = func_sums->get_create (node);
    2714                 :      24360 :   unsigned param_desc_count = streamer_read_uhwi (ib);
    2715                 :      24360 :   if (param_desc_count > 0)
    2716                 :            :     {
    2717                 :      12292 :       vec_safe_reserve_exact (ifs->m_parameters, param_desc_count);
    2718                 :      12292 :       ifs->m_parameters->quick_grow_cleared (param_desc_count);
    2719                 :            :     }
    2720                 :      42714 :   for (unsigned i = 0; i < param_desc_count; i++)
    2721                 :            :     {
    2722                 :      18354 :       isra_param_desc *desc = &(*ifs->m_parameters)[i];
    2723                 :      18354 :       unsigned access_count = streamer_read_uhwi (ib);
    2724                 :      19605 :       for (unsigned j = 0; j < access_count; j++)
    2725                 :            :         {
    2726                 :       1251 :           param_access *acc = ggc_cleared_alloc<param_access> ();
    2727                 :       1251 :           acc->type = stream_read_tree (ib, data_in);
    2728                 :       1251 :           acc->alias_ptr_type = stream_read_tree (ib, data_in);
    2729                 :       1251 :           acc->unit_offset = streamer_read_uhwi (ib);
    2730                 :       1251 :           acc->unit_size = streamer_read_uhwi (ib);
    2731                 :       1251 :           bitpack_d bp = streamer_read_bitpack (ib);
    2732                 :       1251 :           acc->certain = bp_unpack_value (&bp, 1);
    2733                 :       1251 :           vec_safe_push (desc->accesses, acc);
    2734                 :            :         }
    2735                 :      18354 :       desc->param_size_limit = streamer_read_uhwi (ib);
    2736                 :      18354 :       desc->size_reached = streamer_read_uhwi (ib);
    2737                 :      18354 :       bitpack_d bp = streamer_read_bitpack (ib);
    2738                 :      18354 :       desc->locally_unused = bp_unpack_value (&bp, 1);
    2739                 :      18354 :       desc->split_candidate = bp_unpack_value (&bp, 1);
    2740                 :      18354 :       desc->by_ref = bp_unpack_value (&bp, 1);
    2741                 :            :     }
    2742                 :      24360 :   bitpack_d bp = streamer_read_bitpack (ib);
    2743                 :      24360 :   ifs->m_candidate = bp_unpack_value (&bp, 1);
    2744                 :      24360 :   ifs->m_returns_value = bp_unpack_value (&bp, 1);
    2745                 :      24360 :   ifs->m_return_ignored = bp_unpack_value (&bp, 1);
    2746                 :      24360 :   ifs->m_queued = 0;
    2747                 :            : 
    2748                 :     131609 :   for (cgraph_edge *e = node->callees; e; e = e->next_callee)
    2749                 :     107249 :     isra_read_edge_summary (ib, e);
    2750                 :      25326 :   for (cgraph_edge *e = node->indirect_calls; e; e = e->next_callee)
    2751                 :        966 :     isra_read_edge_summary (ib, e);
    2752                 :      24360 : }
    2753                 :            : 
    2754                 :            : /* Read IPA-SRA summaries from a section in file FILE_DATA of length LEN with
    2755                 :            :    data DATA.  TODO: This function was copied almost verbatim from ipa-prop.c,
    2756                 :            :    it should be possible to unify them somehow.  */
    2757                 :            : 
    2758                 :            : static void
    2759                 :       8472 : isra_read_summary_section (struct lto_file_decl_data *file_data,
    2760                 :            :                            const char *data, size_t len)
    2761                 :            : {
    2762                 :       8472 :   const struct lto_function_header *header =
    2763                 :            :     (const struct lto_function_header *) data;
    2764                 :       8472 :   const int cfg_offset = sizeof (struct lto_function_header);
    2765                 :       8472 :   const int main_offset = cfg_offset + header->cfg_size;
    2766                 :       8472 :   const int string_offset = main_offset + header->main_size;
    2767                 :       8472 :   struct data_in *data_in;
    2768                 :       8472 :   unsigned int i;
    2769                 :       8472 :   unsigned int count;
    2770                 :            : 
    2771                 :       8472 :   lto_input_block ib_main ((const char *) data + main_offset,
    2772                 :       8472 :                            header->main_size, file_data->mode_table);
    2773                 :            : 
    2774                 :       8472 :   data_in =
    2775                 :      16944 :     lto_data_in_create (file_data, (const char *) data + string_offset,
    2776                 :       8472 :                         header->string_size, vNULL);
    2777                 :       8472 :   count = streamer_read_uhwi (&ib_main);
    2778                 :            : 
    2779                 :      32832 :   for (i = 0; i < count; i++)
    2780                 :            :     {
    2781                 :      24360 :       unsigned int index;
    2782                 :      24360 :       struct cgraph_node *node;
    2783                 :      24360 :       lto_symtab_encoder_t encoder;
    2784                 :            : 
    2785                 :      24360 :       index = streamer_read_uhwi (&ib_main);
    2786                 :      24360 :       encoder = file_data->symtab_node_encoder;
    2787                 :      48720 :       node = dyn_cast<cgraph_node *> (lto_symtab_encoder_deref (encoder,
    2788                 :            :                                                                 index));
    2789                 :      24360 :       gcc_assert (node->definition);
    2790                 :      24360 :       isra_read_node_info (&ib_main, node, data_in);
    2791                 :            :     }
    2792                 :       8472 :   lto_free_section_data (file_data, LTO_section_ipa_sra, NULL, data,
    2793                 :            :                          len);
    2794                 :       8472 :   lto_data_in_delete (data_in);
    2795                 :       8472 : }
    2796                 :            : 
    2797                 :            : /* Read intraprocedural analysis information into a stream for LTO WPA.  */
    2798                 :            : 
    2799                 :            : static void
    2800                 :       7899 : ipa_sra_read_summary (void)
    2801                 :            : {
    2802                 :       7899 :   struct lto_file_decl_data **file_data_vec = lto_get_file_decl_data ();
    2803                 :       7899 :   struct lto_file_decl_data *file_data;
    2804                 :       7899 :   unsigned int j = 0;
    2805                 :            : 
    2806                 :       7899 :   gcc_checking_assert (!func_sums);
    2807                 :       7899 :   gcc_checking_assert (!call_sums);
    2808                 :       7899 :   func_sums
    2809                 :       7899 :     = (new (ggc_alloc_no_dtor <ipa_sra_function_summaries> ())
    2810                 :       7899 :        ipa_sra_function_summaries (symtab, true));
    2811                 :       7899 :   call_sums = new ipa_sra_call_summaries (symtab);
    2812                 :            : 
    2813                 :      16379 :   while ((file_data = file_data_vec[j++]))
    2814                 :            :     {
    2815                 :       8480 :       size_t len;
    2816                 :       8480 :       const char *data
    2817                 :       8480 :         = lto_get_summary_section_data (file_data, LTO_section_ipa_sra, &len);
    2818                 :       8480 :       if (data)
    2819                 :       8472 :         isra_read_summary_section (file_data, data, len);
    2820                 :            :     }
    2821                 :       7899 : }
    2822                 :            : 
    2823                 :            : /* Dump all IPA-SRA summary data for all cgraph nodes and edges to file F.  */
    2824                 :            : 
    2825                 :            : static void
    2826                 :         37 : ipa_sra_dump_all_summaries (FILE *f)
    2827                 :            : {
    2828                 :         37 :   cgraph_node *node;
    2829                 :        302 :   FOR_EACH_FUNCTION_WITH_GIMPLE_BODY (node)
    2830                 :            :     {
    2831                 :        114 :       fprintf (f, "\nSummary for node %s:\n", node->dump_name ());
    2832                 :            : 
    2833                 :        114 :       isra_func_summary *ifs = func_sums->get (node);
    2834                 :        114 :       if (!ifs)
    2835                 :            :         {
    2836                 :          6 :           fprintf (f, "  Function does not have any associated IPA-SRA "
    2837                 :            :                    "summary\n");
    2838                 :          6 :           continue;
    2839                 :            :         }
    2840                 :        108 :       if (!ifs->m_candidate)
    2841                 :            :         {
    2842                 :          2 :           fprintf (f, "  Not a candidate function\n");
    2843                 :          2 :           continue;
    2844                 :            :         }
    2845                 :        106 :       if (ifs->m_returns_value)
    2846                 :         70 :           fprintf (f, "  Returns value\n");
    2847                 :        106 :       if (vec_safe_is_empty (ifs->m_parameters))
    2848                 :         31 :         fprintf (f, "  No parameter information. \n");
    2849                 :            :       else
    2850                 :        284 :         for (unsigned i = 0; i < ifs->m_parameters->length (); ++i)
    2851                 :            :           {
    2852                 :        209 :             fprintf (f, "  Descriptor for parameter %i:\n", i);
    2853                 :        209 :             dump_isra_param_descriptor (f, &(*ifs->m_parameters)[i]);
    2854                 :            :           }
    2855                 :        106 :       fprintf (f, "\n");
    2856                 :            : 
    2857                 :        106 :       struct cgraph_edge *cs;
    2858                 :        195 :       for (cs = node->callees; cs; cs = cs->next_callee)
    2859                 :            :         {
    2860                 :         89 :           fprintf (f, "  Summary for edge %s->%s:\n", cs->caller->dump_name (),
    2861                 :         89 :                    cs->callee->dump_name ());
    2862                 :         89 :           isra_call_summary *csum = call_sums->get (cs);
    2863                 :         89 :           if (csum)
    2864                 :         89 :             csum->dump (f);
    2865                 :            :           else
    2866                 :          0 :             fprintf (f, "    Call summary is MISSING!\n");
    2867                 :            :         }
    2868                 :            : 
    2869                 :            :     }
    2870                 :         37 :   fprintf (f, "\n\n");
    2871                 :         37 : }
    2872                 :            : 
    2873                 :            : /* Perform function-scope viability tests that can be only made at IPA level
    2874                 :            :    and return false if the function is deemed unsuitable for IPA-SRA.  */
    2875                 :            : 
    2876                 :            : static bool
    2877                 :     596411 : ipa_sra_ipa_function_checks (cgraph_node *node)
    2878                 :            : {
    2879                 :     596411 :   if (!node->can_be_local_p ())
    2880                 :            :     {
    2881                 :     392618 :       if (dump_file)
    2882                 :         63 :         fprintf (dump_file, "Function %s disqualified because it cannot be "
    2883                 :            :                  "made local.\n", node->dump_name ());
    2884                 :     392618 :       return false;
    2885                 :            :     }
    2886                 :     203793 :   if (!node->can_change_signature)
    2887                 :            :     {
    2888                 :          0 :       if (dump_file)
    2889                 :          0 :         fprintf (dump_file, "Function can not change signature.\n");
    2890                 :          0 :       return false;
    2891                 :            :     }
    2892                 :            : 
    2893                 :            :   return true;
    2894                 :            : }
    2895                 :            : 
    2896                 :            : /* Issues found out by check_callers_for_issues.  */
    2897                 :            : 
    2898                 :            : struct caller_issues
    2899                 :            : {
    2900                 :            :   /* The candidate being considered.  */
    2901                 :            :   cgraph_node *candidate;
    2902                 :            :   /* There is a thunk among callers.  */
    2903                 :            :   bool thunk;
    2904                 :            :   /* Call site with no available information.  */
    2905                 :            :   bool unknown_callsite;
    2906                 :            :   /* Call from outside the the candidate's comdat group.  */
    2907                 :            :   bool call_from_outside_comdat;
    2908                 :            :   /* There is a bit-aligned load into one of non-gimple-typed arguments. */
    2909                 :            :   bool bit_aligned_aggregate_argument;
    2910                 :            : };
    2911                 :            : 
    2912                 :            : /* Worker for call_for_symbol_and_aliases, set any flags of passed caller_issues
    2913                 :            :    that apply.  */
    2914                 :            : 
    2915                 :            : static bool
    2916                 :     206995 : check_for_caller_issues (struct cgraph_node *node, void *data)
    2917                 :            : {
    2918                 :     206995 :   struct caller_issues *issues = (struct caller_issues *) data;
    2919                 :            : 
    2920                 :     725386 :   for (cgraph_edge *cs = node->callers; cs; cs = cs->next_caller)
    2921                 :            :     {
    2922                 :     522834 :       if (cs->caller->thunk.thunk_p)
    2923                 :            :         {
    2924                 :          0 :           issues->thunk = true;
    2925                 :            :           /* TODO: We should be able to process at least some types of
    2926                 :            :              thunks.  */
    2927                 :          0 :           return true;
    2928                 :            :         }
    2929                 :     522834 :       if (issues->candidate->calls_comdat_local
    2930                 :          2 :           && issues->candidate->same_comdat_group
    2931                 :     522838 :           && !issues->candidate->in_same_comdat_group_p (cs->caller))
    2932                 :            :         {
    2933                 :          0 :           issues->call_from_outside_comdat = true;
    2934                 :          0 :           return true;
    2935                 :            :         }
    2936                 :            : 
    2937                 :     522834 :       isra_call_summary *csum = call_sums->get (cs);
    2938                 :     522834 :       if (!csum)
    2939                 :            :         {
    2940                 :       4443 :           issues->unknown_callsite = true;
    2941                 :       4443 :           return true;
    2942                 :            :         }
    2943                 :            : 
    2944                 :     518391 :       if (csum->m_bit_aligned_arg)
    2945                 :          0 :         issues->bit_aligned_aggregate_argument = true;
    2946                 :            :     }
    2947                 :            :   return false;
    2948                 :            : }
    2949                 :            : 
    2950                 :            : /* Look at all incoming edges to NODE, including aliases and thunks and look
    2951                 :            :    for problems.  Return true if NODE type should not be modified at all.  */
    2952                 :            : 
    2953                 :            : static bool
    2954                 :     203793 : check_all_callers_for_issues (cgraph_node *node)
    2955                 :            : {
    2956                 :     203793 :   struct caller_issues issues;
    2957                 :     203793 :   memset (&issues, 0, sizeof (issues));
    2958                 :     203793 :   issues.candidate = node;
    2959                 :            : 
    2960                 :     203793 :   node->call_for_symbol_and_aliases (check_for_caller_issues, &issues, true);
    2961                 :     203793 :   if (issues.unknown_callsite)
    2962                 :            :     {
    2963                 :       4443 :       if (dump_file && (dump_flags & TDF_DETAILS))
    2964                 :          0 :         fprintf (dump_file, "A call of %s has not been analyzed.  Disabling "
    2965                 :            :                  "all modifications.\n", node->dump_name ());
    2966                 :       4443 :       return true;
    2967                 :            :     }
    2968                 :            :   /* TODO: We should be able to process at least some types of thunks.  */
    2969                 :     199350 :   if (issues.thunk)
    2970                 :            :     {
    2971                 :          0 :       if (dump_file && (dump_flags & TDF_DETAILS))
    2972                 :          0 :         fprintf (dump_file, "A call of %s is through thunk, which are not"
    2973                 :            :                  " handled yet.  Disabling all modifications.\n",
    2974                 :            :                  node->dump_name ());
    2975                 :          0 :       return true;
    2976                 :            :     }
    2977                 :     199350 :   if (issues.call_from_outside_comdat)
    2978                 :            :     {
    2979                 :          0 :       if (dump_file)
    2980                 :          0 :         fprintf (dump_file, "Function would become private comdat called "
    2981                 :            :                  "outside of its comdat group.\n");
    2982                 :          0 :       return true;
    2983                 :            :     }
    2984                 :            : 
    2985                 :     199350 :   if (issues.bit_aligned_aggregate_argument)
    2986                 :            :     {
    2987                 :            :       /* Let's only remove parameters/return values from such functions.
    2988                 :            :          TODO: We could only prevent splitting the problematic parameters if
    2989                 :            :          anybody thinks it is worth it.  */
    2990                 :          0 :       if (dump_file && (dump_flags & TDF_DETAILS))
    2991                 :          0 :         fprintf (dump_file, "A call of %s has bit-aligned aggregate argument,"
    2992                 :            :                  " disabling parameter splitting.\n", node->dump_name ());
    2993                 :            : 
    2994                 :          0 :       isra_func_summary *ifs = func_sums->get (node);
    2995                 :          0 :       gcc_checking_assert (ifs);
    2996                 :          0 :       unsigned param_count = vec_safe_length (ifs->m_parameters);
    2997                 :          0 :       for (unsigned i = 0; i < param_count; i++)
    2998                 :          0 :         (*ifs->m_parameters)[i].split_candidate = false;
    2999                 :            :     }
    3000                 :            :   return false;
    3001                 :            : }
    3002                 :            : 
    3003                 :            : /* Find the access with corresponding OFFSET and SIZE among accesses in
    3004                 :            :    PARAM_DESC and return it or NULL if such an access is not there.  */
    3005                 :            : 
    3006                 :            : static param_access *
    3007                 :       8958 : find_param_access (isra_param_desc *param_desc, unsigned offset, unsigned size)
    3008                 :            : {
    3009                 :       8958 :   unsigned pclen = vec_safe_length (param_desc->accesses);
    3010                 :            : 
    3011                 :            :   /* The search is linear but the number of stored accesses is bound by
    3012                 :            :      PARAM_IPA_SRA_MAX_REPLACEMENTS, so most probably 8.  */
    3013                 :            : 
    3014                 :       9024 :   for (unsigned i = 0; i < pclen; i++)
    3015                 :       8282 :     if ((*param_desc->accesses)[i]->unit_offset == offset
    3016                 :       8282 :         && (*param_desc->accesses)[i]->unit_size == size)
    3017                 :       8958 :       return (*param_desc->accesses)[i];
    3018                 :            : 
    3019                 :            :   return NULL;
    3020                 :            : }
    3021                 :            : 
    3022                 :            : /* Return iff the total size of definite replacement SIZE would violate the
    3023                 :            :    limit set for it in PARAM.  */
    3024                 :            : 
    3025                 :            : static bool
    3026                 :       7036 : size_would_violate_limit_p (isra_param_desc *desc, unsigned size)
    3027                 :            : {
    3028                 :       7036 :   unsigned limit = desc->param_size_limit;
    3029                 :          0 :   if (size > limit
    3030                 :       7027 :       || (!desc->by_ref && size == limit))
    3031                 :          0 :     return true;
    3032                 :            :   return false;
    3033                 :            : }
    3034                 :            : 
    3035                 :            : /* Increase reached size of DESC by SIZE or disqualify it if it would violate
    3036                 :            :    the set limit.  IDX is the parameter number which is dumped when
    3037                 :            :    disqualifying.  */
    3038                 :            : 
    3039                 :            : static void
    3040                 :       6355 : bump_reached_size (isra_param_desc *desc, unsigned size, unsigned idx)
    3041                 :            : {
    3042                 :       6355 :   unsigned after = desc->size_reached + size;
    3043                 :       6355 :   if (size_would_violate_limit_p (desc, after))
    3044                 :            :     {
    3045                 :       6336 :       if (dump_file && (dump_flags & TDF_DETAILS))
    3046                 :          0 :         fprintf (dump_file, "    ...size limit reached, disqualifying "
    3047                 :            :                  "candidate parameter %u\n", idx);
    3048                 :       6336 :       desc->split_candidate = false;
    3049                 :       6336 :       return;
    3050                 :            :     }
    3051                 :         19 :   desc->size_reached = after;
    3052                 :            : }
    3053                 :            : 
    3054                 :            : /* Take all actions required to deal with an edge CS that represents a call to
    3055                 :            :    an unknown or un-analyzed function, for both parameter removal and
    3056                 :            :    splitting.  */
    3057                 :            : 
    3058                 :            : static void
    3059                 :     182350 : process_edge_to_unknown_caller (cgraph_edge *cs)
    3060                 :            : {
    3061                 :     182350 :   isra_func_summary *from_ifs = func_sums->get (cs->caller);
    3062                 :     182350 :   gcc_checking_assert (from_ifs);
    3063                 :     182350 :   isra_call_summary *csum = call_sums->get (cs);
    3064                 :            : 
    3065                 :     182350 :   if (dump_file && (dump_flags & TDF_DETAILS))
    3066                 :          2 :     fprintf (dump_file, "Processing an edge to an unknown caller from %s:\n",
    3067                 :          2 :              cs->caller->dump_name ());
    3068                 :            : 
    3069                 :     182350 :   unsigned args_count = csum->m_arg_flow.length ();
    3070                 :     370659 :   for (unsigned i = 0; i < args_count; i++)
    3071                 :            :     {
    3072                 :     188309 :       isra_param_flow *ipf = &csum->m_arg_flow[i];
    3073                 :            : 
    3074                 :     188309 :      if (ipf->pointer_pass_through)
    3075                 :            :        {
    3076                 :      28778 :          isra_param_desc *param_desc
    3077                 :      28778 :            = &(*from_ifs->m_parameters)[get_single_param_flow_source (ipf)];
    3078                 :      28778 :          param_desc->locally_unused = false;
    3079                 :      28778 :          param_desc->split_candidate = false;
    3080                 :      28778 :         continue;
    3081                 :            :        }
    3082                 :     159531 :       if (ipf->aggregate_pass_through)
    3083                 :            :         {
    3084                 :       4904 :           unsigned idx = get_single_param_flow_source (ipf);
    3085                 :       4904 :           isra_param_desc *param_desc = &(*from_ifs->m_parameters)[idx];
    3086                 :            : 
    3087                 :       4904 :           param_desc->locally_unused = false;
    3088                 :       4904 :           if (!param_desc->split_candidate)
    3089                 :       1183 :             continue;
    3090                 :       3721 :           gcc_assert (!param_desc->by_ref);
    3091                 :       7442 :           param_access *pacc = find_param_access (param_desc, ipf->unit_offset,
    3092                 :       3721 :                                                   ipf->unit_size);
    3093                 :       3721 :           gcc_checking_assert (pacc);
    3094                 :       3721 :           pacc->certain = true;
    3095                 :       3721 :           if (overlapping_certain_accesses_p (param_desc, NULL))
    3096                 :            :             {
    3097                 :        375 :               if (dump_file && (dump_flags & TDF_DETAILS))
    3098                 :          0 :                 fprintf (dump_file, "    ...leading to overlap, "
    3099                 :            :                          " disqualifying candidate parameter %u\n",
    3100                 :            :                          idx);
    3101                 :        375 :               param_desc->split_candidate = false;
    3102                 :            :             }
    3103                 :            :           else
    3104                 :       3346 :             bump_reached_size (param_desc, pacc->unit_size, idx);
    3105                 :       3721 :           ipf->aggregate_pass_through = false;
    3106                 :       3721 :           continue;
    3107                 :            :         }
    3108                 :            : 
    3109                 :     204749 :       for (int j = 0; j < ipf->length; j++)
    3110                 :            :         {
    3111                 :      50122 :           int input_idx = ipf->inputs[j];
    3112                 :      50122 :           (*from_ifs->m_parameters)[input_idx].locally_unused = false;
    3113                 :            :         }
    3114                 :            :     }
    3115                 :     182350 : }
    3116                 :            : 
    3117                 :            : /* Propagate parameter removal information through cross-SCC edge CS,
    3118                 :            :    i.e. decrease the use count in the caller parameter descriptor for each use
    3119                 :            :    in this call.  */
    3120                 :            : 
    3121                 :            : static void
    3122                 :     302724 : param_removal_cross_scc_edge (cgraph_edge *cs)
    3123                 :            : {
    3124                 :     302724 :   enum availability availability;
    3125                 :     302724 :   cgraph_node *callee = cs->callee->function_symbol (&availability);
    3126                 :     302724 :   isra_func_summary *to_ifs = func_sums->get (callee);
    3127                 :     302724 :   if (!to_ifs || !to_ifs->m_candidate
    3128                 :     133940 :       || (availability < AVAIL_AVAILABLE)
    3129                 :     294458 :       || vec_safe_is_empty (to_ifs->m_parameters))
    3130                 :            :     {
    3131                 :     173630 :       process_edge_to_unknown_caller (cs);
    3132                 :     173630 :       return;
    3133                 :            :     }
    3134                 :     129094 :   isra_func_summary *from_ifs = func_sums->get (cs->caller);
    3135                 :     129094 :   gcc_checking_assert (from_ifs);
    3136                 :            : 
    3137                 :     129094 :   isra_call_summary *csum = call_sums->get (cs);
    3138                 :     129094 :   unsigned args_count = csum->m_arg_flow.length ();
    3139                 :     129094 :   unsigned param_count = vec_safe_length (to_ifs->m_parameters);
    3140                 :            : 
    3141                 :     335724 :   for (unsigned i = 0; i < args_count; i++)
    3142                 :            :     {
    3143                 :     206630 :       bool unused_in_callee;
    3144                 :     206630 :       if (i < param_count)
    3145                 :     206630 :         unused_in_callee = (*to_ifs->m_parameters)[i].locally_unused;
    3146                 :            :       else
    3147                 :            :         unused_in_callee = false;
    3148                 :            : 
    3149                 :     206630 :       if (!unused_in_callee)
    3150                 :            :         {
    3151                 :     186125 :           isra_param_flow *ipf = &csum->m_arg_flow[i];
    3152                 :     285276 :           for (int j = 0; j < ipf->length; j++)
    3153                 :            :             {
    3154                 :      99151 :               int input_idx = ipf->inputs[j];
    3155                 :      99151 :               (*from_ifs->m_parameters)[input_idx].locally_unused = false;
    3156                 :            :             }
    3157                 :            :         }
    3158                 :            :     }
    3159                 :            : }
    3160                 :            : 
    3161                 :            : /* Unless it is already there, push NODE which is also described by IFS to
    3162                 :            :    STACK.  */
    3163                 :            : 
    3164                 :            : static void
    3165                 :      82504 : isra_push_node_to_stack (cgraph_node *node, isra_func_summary *ifs,
    3166                 :            :                     vec<cgraph_node *> *stack)
    3167                 :            : {
    3168                 :          0 :   if (!ifs->m_queued)
    3169                 :            :     {
    3170                 :      81921 :       ifs->m_queued = true;
    3171                 :      81921 :       stack->safe_push (node);
    3172                 :            :     }
    3173                 :          0 : }
    3174                 :            : 
    3175                 :            : /* If parameter with index INPUT_IDX is marked as locally unused, mark it as
    3176                 :            :    used and push CALLER on STACK.  */
    3177                 :            : 
    3178                 :            : static void
    3179                 :      19617 : isra_mark_caller_param_used (isra_func_summary *from_ifs, int input_idx,
    3180                 :            :                              cgraph_node *caller, vec<cgraph_node *> *stack)
    3181                 :            : {
    3182                 :      19617 :   if ((*from_ifs->m_parameters)[input_idx].locally_unused)
    3183                 :            :     {
    3184                 :       2431 :       (*from_ifs->m_parameters)[input_idx].locally_unused = false;
    3185                 :       4279 :       isra_push_node_to_stack (caller, from_ifs, stack);
    3186                 :            :     }
    3187                 :      19617 : }
    3188                 :            : 
    3189                 :            : 
    3190                 :            : /* Propagate information that any parameter is not used only locally within a
    3191                 :            :    SCC across CS to the caller, which must be in the same SCC as the
    3192                 :            :    callee. Push any callers that need to be re-processed to STACK.  */
    3193                 :            : 
    3194                 :            : static void
    3195                 :      24635 : propagate_used_across_scc_edge (cgraph_edge *cs, vec<cgraph_node *> *stack)
    3196                 :            : {
    3197                 :      24635 :   isra_func_summary *from_ifs = func_sums->get (cs->caller);
    3198                 :      24635 :   if (!from_ifs || vec_safe_is_empty (from_ifs->m_parameters))
    3199                 :      11496 :     return;
    3200                 :            : 
    3201                 :      13139 :   isra_call_summary *csum = call_sums->get (cs);
    3202                 :      13139 :   gcc_checking_assert (csum);
    3203                 :      13139 :   unsigned args_count = csum->m_arg_flow.length ();
    3204                 :      13139 :   enum availability availability;
    3205                 :      13139 :   cgraph_node *callee = cs->callee->function_symbol (&availability);
    3206                 :      13139 :   isra_func_summary *to_ifs = func_sums->get (callee);
    3207                 :            : 
    3208                 :         12 :   unsigned param_count
    3209                 :      13127 :     = (to_ifs && (availability >= AVAIL_AVAILABLE))
    3210                 :      26156 :     ? vec_safe_length (to_ifs->m_parameters) : 0;
    3211                 :      43970 :   for (unsigned i = 0; i < args_count; i++)
    3212                 :            :     {
    3213                 :      33163 :       if (i < param_count
    3214                 :      30831 :           && (*to_ifs->m_parameters)[i].locally_unused)
    3215                 :       2332 :             continue;
    3216                 :            : 
    3217                 :            :       /* The argument is needed in the callee it, we must mark the parameter as
    3218                 :            :          used also in the caller and its callers within this SCC.  */
    3219                 :      28499 :       isra_param_flow *ipf = &csum->m_arg_flow[i];
    3220                 :      48116 :       for (int j = 0; j < ipf->length; j++)
    3221                 :            :         {
    3222                 :      19617 :           int input_idx = ipf->inputs[j];
    3223                 :      19617 :           isra_mark_caller_param_used (from_ifs, input_idx, cs->caller, stack);
    3224                 :            :         }
    3225                 :            :     }
    3226                 :            : }
    3227                 :            : 
    3228                 :            : /* Propagate information that any parameter is not used only locally within a
    3229                 :            :    SCC (i.e. is used also elsewhere) to all callers of NODE that are in the
    3230                 :            :    same SCC. Push any callers that need to be re-processed to STACK.  */
    3231                 :            : 
    3232                 :            : static bool
    3233                 :     924918 : propagate_used_to_scc_callers (cgraph_node *node, void *data)
    3234                 :            : {
    3235                 :     924918 :   vec<cgraph_node *> *stack = (vec<cgraph_node *> *) data;
    3236                 :     924918 :   cgraph_edge *cs;
    3237                 :    2080840 :   for (cs = node->callers; cs; cs = cs->next_caller)
    3238                 :    1155920 :     if (ipa_edge_within_scc (cs))
    3239                 :      24635 :       propagate_used_across_scc_edge (cs, stack);
    3240                 :     924918 :   return false;
    3241                 :            : }
    3242                 :            : 
    3243                 :            : /* Return true iff all certain accesses in ARG_DESC are also present as
    3244                 :            :    certain accesses in PARAM_DESC.  */
    3245                 :            : 
    3246                 :            : static bool
    3247                 :        997 : all_callee_accesses_present_p (isra_param_desc *param_desc,
    3248                 :            :                                isra_param_desc *arg_desc)
    3249                 :            : {
    3250                 :        997 :   unsigned aclen = vec_safe_length (arg_desc->accesses);
    3251                 :       1015 :   for (unsigned j = 0; j < aclen; j++)
    3252                 :            :     {
    3253                 :        760 :       param_access *argacc = (*arg_desc->accesses)[j];
    3254                 :        760 :       if (!argacc->certain)
    3255                 :          0 :         continue;
    3256                 :       1520 :       param_access *pacc = find_param_access (param_desc, argacc->unit_offset,
    3257                 :        760 :                                               argacc->unit_size);
    3258                 :        760 :       if (!pacc || !pacc->certain)
    3259                 :            :         return false;
    3260                 :            :     }
    3261                 :            :   return true;
    3262                 :            : }
    3263                 :            : 
    3264                 :            : /* Type internal to function pull_accesses_from_callee.  Unfortunately gcc 4.8
    3265                 :            :    does not allow instantiating an auto_vec with a type defined within a
    3266                 :            :    function so it is a global type.   */
    3267                 :            : enum acc_prop_kind {ACC_PROP_DONT, ACC_PROP_COPY, ACC_PROP_CERTAIN};
    3268                 :            : 
    3269                 :            : 
    3270                 :            : /* Attempt to propagate all definite accesses from ARG_DESC to PARAM_DESC,
    3271                 :            :    (which belongs to CALLER) if they would not violate some constraint there.
    3272                 :            :    If successful, return NULL, otherwise return the string reason for failure
    3273                 :            :    (which can be written to the dump file).  DELTA_OFFSET is the known offset
    3274                 :            :    of the actual argument withing the formal parameter (so of ARG_DESCS within
    3275                 :            :    PARAM_DESCS), ARG_SIZE is the size of the actual argument or zero, if not
    3276                 :            :    known. In case of success, set *CHANGE_P to true if propagation actually
    3277                 :            :    changed anything.  */
    3278                 :            : 
    3279                 :            : static const char *
    3280                 :       1173 : pull_accesses_from_callee (cgraph_node *caller, isra_param_desc *param_desc,
    3281                 :            :                            isra_param_desc *arg_desc,
    3282                 :            :                            unsigned delta_offset, unsigned arg_size,
    3283                 :            :                            bool *change_p)
    3284                 :            : {
    3285                 :       1173 :   unsigned pclen = vec_safe_length (param_desc->accesses);
    3286                 :       1173 :   unsigned aclen = vec_safe_length (arg_desc->accesses);
    3287                 :       1173 :   unsigned prop_count = 0;
    3288                 :       1173 :   unsigned prop_size = 0;
    3289                 :       1173 :   bool change = false;
    3290                 :            : 
    3291                 :       2346 :   auto_vec <enum acc_prop_kind, 8> prop_kinds (aclen);
    3292                 :       3687 :   for (unsigned j = 0; j < aclen; j++)
    3293                 :            :     {
    3294                 :       2533 :       param_access *argacc = (*arg_desc->accesses)[j];
    3295                 :       2533 :       prop_kinds.safe_push (ACC_PROP_DONT);
    3296                 :            : 
    3297                 :       2533 :       if (arg_size > 0
    3298                 :       2191 :           && argacc->unit_offset + argacc->unit_size > arg_size)
    3299                 :            :         return "callee access outsize size boundary";
    3300                 :            : 
    3301                 :       2533 :       if (!argacc->certain)
    3302                 :        254 :         continue;
    3303                 :            : 
    3304                 :       2279 :       unsigned offset = argacc->unit_offset + delta_offset;
    3305                 :            :       /* Given that accesses are initially stored according to increasing
    3306                 :            :          offset and decreasing size in case of equal offsets, the following
    3307                 :            :          searches could be written more efficiently if we kept the ordering
    3308                 :            :          when copying. But the number of accesses is capped at
    3309                 :            :          PARAM_IPA_SRA_MAX_REPLACEMENTS (so most likely 8) and the code gets
    3310                 :            :          messy quickly, so let's improve on that only if necessary.  */
    3311                 :            : 
    3312                 :       2279 :       bool exact_match = false;
    3313                 :       6865 :       for (unsigned i = 0; i < pclen; i++)
    3314                 :            :         {
    3315                 :            :           /* Check for overlaps.  */
    3316                 :       4605 :           param_access *pacc = (*param_desc->accesses)[i];
    3317                 :       4605 :           if (pacc->unit_offset == offset
    3318                 :       1917 :               && pacc->unit_size == argacc->unit_size)
    3319                 :            :             {
    3320                 :       1038 :               if (argacc->alias_ptr_type != pacc->alias_ptr_type
    3321                 :       1038 :                   || !types_compatible_p (argacc->type, pacc->type))
    3322                 :         18 :                 return "propagated access types would not match existing ones";
    3323                 :            : 
    3324                 :       1020 :               exact_match = true;
    3325                 :       1020 :               if (!pacc->certain)
    3326                 :            :                 {
    3327                 :          0 :                   prop_kinds[j] = ACC_PROP_CERTAIN;
    3328                 :          0 :                   prop_size += argacc->unit_size;
    3329                 :          0 :                   change = true;
    3330                 :            :                 }
    3331                 :       1020 :               continue;
    3332                 :            :             }
    3333                 :            : 
    3334                 :       3567 :           if (offset < pacc->unit_offset + pacc->unit_size
    3335                 :       2803 :               && offset + argacc->unit_size > pacc->unit_offset)
    3336                 :            :             {
    3337                 :            :               /* None permissible with load accesses, possible to fit into
    3338                 :            :                  argument ones.  */
    3339                 :       1938 :               if (pacc->certain
    3340                 :       1938 :                   || offset < pacc->unit_offset
    3341                 :       1937 :                   || (offset + argacc->unit_size
    3342                 :            :                       > pacc->unit_offset + pacc->unit_size))
    3343                 :            :                 return "a propagated access would conflict in caller";
    3344                 :            :             }
    3345                 :            :         }
    3346                 :            : 
    3347                 :       2260 :       if (!exact_match)
    3348                 :            :         {
    3349                 :       1240 :           prop_kinds[j] = ACC_PROP_COPY;
    3350                 :       1240 :           prop_count++;
    3351                 :       1240 :           prop_size += argacc->unit_size;
    3352                 :       1240 :           change = true;
    3353                 :            :         }
    3354                 :            :     }
    3355                 :            : 
    3356                 :       1154 :     if (!change)
    3357                 :            :       return NULL;
    3358                 :            : 
    3359                 :        681 :     if ((prop_count + pclen
    3360                 :        681 :          > (unsigned) opt_for_fn (caller->decl, param_ipa_sra_max_replacements))
    3361                 :       1854 :         || size_would_violate_limit_p (param_desc,
    3362                 :        681 :                                        param_desc->size_reached + prop_size))
    3363                 :            :       return "propagating accesses would violate the count or size limit";
    3364                 :            : 
    3365                 :        664 :   *change_p = true;
    3366                 :       2096 :   for (unsigned j = 0; j < aclen; j++)
    3367                 :            :     {
    3368                 :       1432 :       if (prop_kinds[j] == ACC_PROP_COPY)
    3369                 :            :         {
    3370                 :       1214 :           param_access *argacc = (*arg_desc->accesses)[j];
    3371                 :            : 
    3372                 :       1214 :           param_access *copy = ggc_cleared_alloc<param_access> ();
    3373                 :       1214 :           copy->unit_offset = argacc->unit_offset + delta_offset;
    3374                 :       1214 :           copy->unit_size = argacc->unit_size;
    3375                 :       1214 :           copy->type = argacc->type;
    3376                 :       1214 :           copy->alias_ptr_type = argacc->alias_ptr_type;
    3377                 :       1214 :           copy->certain = true;
    3378                 :       1214 :           vec_safe_push (param_desc->accesses, copy);
    3379                 :            :         }
    3380                 :        218 :       else if (prop_kinds[j] == ACC_PROP_CERTAIN)
    3381                 :            :         {
    3382                 :          0 :           param_access *argacc = (*arg_desc->accesses)[j];
    3383                 :          0 :           param_access *csp
    3384                 :          0 :             = find_param_access (param_desc, argacc->unit_offset + delta_offset,
    3385                 :          0 :                                  argacc->unit_size);
    3386                 :          0 :           csp->certain = true;
    3387                 :            :         }
    3388                 :            :     }
    3389                 :            : 
    3390                 :        664 :   param_desc->size_reached += prop_size;
    3391                 :            : 
    3392                 :        664 :   return NULL;
    3393                 :            : }
    3394                 :            : 
    3395                 :            : /* Propagate parameter splitting information through call graph edge CS.
    3396                 :            :    Return true if any changes that might need to be propagated within SCCs have
    3397                 :            :    been made.  The function also clears the aggregate_pass_through and
    3398                 :            :    pointer_pass_through in call summaries which do not need to be processed
    3399                 :            :    again if this CS is revisited when iterating while changes are propagated
    3400                 :            :    within an SCC.  */
    3401                 :            : 
    3402                 :            : static bool
    3403                 :     684258 : param_splitting_across_edge (cgraph_edge *cs)
    3404                 :            : {
    3405                 :     684258 :   bool res = false;
    3406                 :     684258 :   bool cross_scc = !ipa_edge_within_scc (cs);
    3407                 :     684258 :   enum availability availability;
    3408                 :     684258 :   cgraph_node *callee = cs->callee->function_symbol (&availability);
    3409                 :     684258 :   isra_func_summary *from_ifs = func_sums->get (cs->caller);
    3410                 :     684258 :   gcc_checking_assert (from_ifs && from_ifs->m_parameters);
    3411                 :            : 
    3412                 :     684258 :   isra_call_summary *csum = call_sums->get (cs);
    3413                 :     684258 :   gcc_checking_assert (csum);
    3414                 :     684258 :   unsigned args_count = csum->m_arg_flow.length ();
    3415                 :     684258 :   isra_func_summary *to_ifs = func_sums->get (callee);
    3416                 :     303587 :   unsigned param_count
    3417                 :     380671 :     = ((to_ifs && to_ifs->m_candidate && (availability >= AVAIL_AVAILABLE))
    3418                 :     702985 :        ? vec_safe_length (to_ifs->m_parameters)
    3419                 :     684258 :        : 0);
    3420                 :            : 
    3421                 :     684258 :   if (dump_file && (dump_flags & TDF_DETAILS))
    3422                 :          4 :     fprintf (dump_file, "Splitting across %s->%s:\n",
    3423                 :          2 :              cs->caller->dump_name (), callee->dump_name ());
    3424                 :            : 
    3425                 :            :   unsigned i;
    3426                 :    1103960 :   for (i = 0; (i < args_count) && (i < param_count); i++)
    3427                 :            :     {
    3428                 :     419704 :       isra_param_desc *arg_desc = &(*to_ifs->m_parameters)[i];
    3429                 :     419704 :       isra_param_flow *ipf = &csum->m_arg_flow[i];
    3430                 :            : 
    3431                 :     419704 :       if (arg_desc->locally_unused)
    3432                 :            :         {
    3433                 :      34060 :           if (dump_file && (dump_flags & TDF_DETAILS))
    3434                 :          0 :             fprintf (dump_file, "    ->%u: unused in callee\n", i);
    3435                 :      34060 :           ipf->pointer_pass_through = false;
    3436                 :      34060 :           continue;
    3437                 :            :         }
    3438                 :            : 
    3439                 :     385644 :       if (ipf->pointer_pass_through)
    3440                 :            :         {
    3441                 :      71100 :           int idx = get_single_param_flow_source (ipf);
    3442                 :      71100 :           isra_param_desc *param_desc = &(*from_ifs->m_parameters)[idx];
    3443                 :      71100 :           if (!param_desc->split_candidate)
    3444                 :      50101 :             continue;
    3445                 :      20999 :           gcc_assert (param_desc->by_ref);
    3446                 :            : 
    3447                 :      20999 :           if (!arg_desc->split_candidate || !arg_desc->by_ref)
    3448                 :            :             {
    3449                 :      19796 :               if (dump_file && (dump_flags & TDF_DETAILS))
    3450                 :          0 :                 fprintf (dump_file, "  %u->%u: not candidate or not by "
    3451                 :            :                          "reference in callee\n", idx, i);
    3452                 :      19796 :               param_desc->split_candidate = false;
    3453                 :      19796 :               ipf->pointer_pass_through = false;
    3454                 :      19796 :               res = true;
    3455                 :            :             }
    3456                 :       1203 :           else if (!ipf->safe_to_import_accesses)
    3457                 :            :             {
    3458                 :        997 :               if (!all_callee_accesses_present_p (param_desc, arg_desc))
    3459                 :            :                 {
    3460                 :        742 :                   if (dump_file && (dump_flags & TDF_DETAILS))
    3461                 :          0 :                     fprintf (dump_file, "  %u->%u: cannot import accesses.\n",
    3462                 :            :                              idx, i);
    3463                 :        742 :                   param_desc->split_candidate = false;
    3464                 :        742 :                   ipf->pointer_pass_through = false;
    3465                 :        742 :                   res = true;
    3466                 :            : 
    3467                 :            :                 }
    3468                 :            :               else
    3469                 :            :                 {
    3470                 :        255 :                   if (dump_file && (dump_flags & TDF_DETAILS))
    3471                 :          0 :                     fprintf (dump_file, "  %u->%u: verified callee accesses "
    3472                 :            :                              "present.\n", idx, i);
    3473                 :        255 :                   if (cross_scc)
    3474                 :          5 :                     ipf->pointer_pass_through = false;
    3475                 :            :                 }
    3476                 :            :             }
    3477                 :            :           else
    3478                 :            :             {
    3479                 :        206 :               const char *pull_failure
    3480                 :        206 :                 = pull_accesses_from_callee (cs->caller, param_desc, arg_desc,
    3481                 :            :                                              0, 0, &res);
    3482                 :        206 :               if (pull_failure)
    3483                 :            :                 {
    3484                 :          9 :                   if (dump_file && (dump_flags & TDF_DETAILS))
    3485                 :          0 :                     fprintf (dump_file, "  %u->%u: by_ref access pull "
    3486                 :            :                              "failed: %s.\n", idx, i, pull_failure);
    3487                 :          9 :                   param_desc->split_candidate = false;
    3488                 :          9 :                   ipf->pointer_pass_through = false;
    3489                 :          9 :                   res = true;
    3490                 :            :                 }
    3491                 :            :               else
    3492                 :            :                 {
    3493                 :        197 :                   if (dump_file && (dump_flags & TDF_DETAILS))
    3494                 :          0 :                     fprintf (dump_file, "  %u->%u: by_ref access pull "
    3495                 :            :                              "succeeded.\n", idx, i);
    3496                 :        197 :                   if (cross_scc)
    3497                 :        195 :                     ipf->pointer_pass_through = false;
    3498                 :            :                 }
    3499                 :            :             }
    3500                 :            :         }
    3501                 :     314544 :       else if (ipf->aggregate_pass_through)
    3502                 :            :         {
    3503                 :      15456 :           int idx = get_single_param_flow_source (ipf);
    3504                 :      15456 :           isra_param_desc *param_desc = &(*from_ifs->m_parameters)[idx];
    3505                 :      15456 :           if (!param_desc->split_candidate)
    3506                 :      10979 :             continue;
    3507                 :       4477 :           gcc_assert (!param_desc->by_ref);
    3508                 :       8954 :           param_access *pacc = find_param_access (param_desc, ipf->unit_offset,
    3509                 :       4477 :                                                   ipf->unit_size);
    3510                 :       4477 :           gcc_checking_assert (pacc);
    3511                 :            : 
    3512                 :       4477 :           if (pacc->certain)
    3513                 :            :             {
    3514                 :          0 :               if (dump_file && (dump_flags & TDF_DETAILS))
    3515                 :          0 :                 fprintf (dump_file, "  %u->%u: already certain\n", idx, i);
    3516                 :          0 :               ipf->aggregate_pass_through = false;
    3517                 :            :             }
    3518                 :       4477 :           else if (!arg_desc->split_candidate || arg_desc->by_ref)
    3519                 :            :             {
    3520                 :       3510 :               if (dump_file && (dump_flags & TDF_DETAILS))
    3521                 :          0 :                 fprintf (dump_file, "  %u->%u: not candidate or by "
    3522                 :            :                          "reference in callee\n", idx, i);
    3523                 :            : 
    3524                 :       3510 :               pacc->certain = true;
    3525                 :       3510 :               if (overlapping_certain_accesses_p (param_desc, NULL))
    3526                 :            :                 {
    3527                 :        501 :                   if (dump_file && (dump_flags & TDF_DETAILS))
    3528                 :          0 :                     fprintf (dump_file, "    ...leading to overlap, "
    3529                 :            :                              " disqualifying candidate parameter %u\n",
    3530                 :            :                              idx);
    3531                 :        501 :                   param_desc->split_candidate = false;
    3532                 :            :                 }
    3533                 :            :               else
    3534                 :       3009 :                 bump_reached_size (param_desc, pacc->unit_size, idx);
    3535                 :            : 
    3536                 :       3510 :               ipf->aggregate_pass_through = false;
    3537                 :       3510 :               res = true;
    3538                 :            :             }
    3539                 :            :           else
    3540                 :            :             {
    3541                 :        967 :               const char *pull_failure
    3542                 :        967 :                 = pull_accesses_from_callee (cs->caller, param_desc, arg_desc,
    3543                 :            :                                              ipf->unit_offset,
    3544                 :            :                                              ipf->unit_size, &res);
    3545                 :        967 :               if (pull_failure)
    3546                 :            :                 {
    3547                 :         27 :                   if (dump_file && (dump_flags & TDF_DETAILS))
    3548                 :          0 :                     fprintf (dump_file, "  %u->%u: arg access pull "
    3549                 :            :                              "failed: %s.\n", idx, i, pull_failure);
    3550                 :            : 
    3551                 :         27 :                   ipf->aggregate_pass_through = false;
    3552                 :         27 :                   pacc->certain = true;
    3553                 :            : 
    3554                 :         27 :                   if (overlapping_certain_accesses_p (param_desc, NULL))
    3555                 :            :                     {
    3556                 :         27 :                       if (dump_file && (dump_flags & TDF_DETAILS))
    3557                 :          0 :                         fprintf (dump_file, "    ...leading to overlap, "
    3558                 :            :                                  " disqualifying candidate parameter %u\n",
    3559                 :            :                                  idx);
    3560                 :         27 :                       param_desc->split_candidate = false;
    3561                 :            :                     }
    3562                 :            :                   else
    3563                 :          0 :                     bump_reached_size (param_desc, pacc->unit_size, idx);
    3564                 :            : 
    3565                 :         27 :                   res = true;
    3566                 :            :                 }
    3567                 :            :               else
    3568                 :            :                 {
    3569                 :        940 :                   if (dump_file && (dump_flags & TDF_DETAILS))
    3570                 :          0 :                     fprintf (dump_file, "  %u->%u: arg access pull "
    3571                 :            :                              "succeeded.\n", idx, i);
    3572                 :        940 :                   if (cross_scc)
    3573                 :        841 :                     ipf->aggregate_pass_through = false;
    3574                 :            :                 }
    3575                 :            :             }
    3576                 :            :         }
    3577                 :            :     }
    3578                 :            : 
    3579                 :            :   /* Handle argument-parameter count mismatches. */
    3580                 :     927888 :   for (; (i < args_count); i++)
    3581                 :            :     {
    3582                 :     243630 :       isra_param_flow *ipf = &csum->m_arg_flow[i];
    3583                 :            : 
    3584                 :     243630 :       if (ipf->pointer_pass_through || ipf->aggregate_pass_through)
    3585                 :            :         {
    3586                 :      33032 :           int idx = get_single_param_flow_source (ipf);
    3587                 :      33032 :           ipf->pointer_pass_through = false;
    3588                 :      33032 :           ipf->aggregate_pass_through = false;
    3589                 :      33032 :           isra_param_desc *param_desc = &(*from_ifs->m_parameters)[idx];
    3590                 :      33032 :           if (!param_desc->split_candidate)
    3591                 :      32965 :             continue;
    3592                 :            : 
    3593                 :         67 :           if (dump_file && (dump_flags & TDF_DETAILS))
    3594                 :          0 :             fprintf (dump_file, "  %u->%u: no corresponding formal parameter\n",
    3595                 :            :                      idx, i);
    3596                 :         67 :           param_desc->split_candidate = false;
    3597                 :         67 :           res = true;
    3598                 :            :         }
    3599                 :            :     }
    3600                 :     684258 :   return res;
    3601                 :            : }
    3602                 :            : 
    3603                 :            : /* Worker for call_for_symbol_and_aliases, look at all callers and if all their
    3604                 :            :    callers ignore the return value, or come from the same SCC and use the
    3605                 :            :    return value only to compute their return value, return false, otherwise
    3606                 :            :    return true.  */
    3607                 :            : 
    3608                 :            : static bool
    3609                 :      95378 : retval_used_p (cgraph_node *node, void *)
    3610                 :            : {
    3611                 :     134821 :   for (cgraph_edge *cs = node->callers; cs; cs = cs->next_caller)
    3612                 :            :     {
    3613                 :     119283 :       isra_call_summary *csum = call_sums->get (cs);
    3614                 :     119283 :       gcc_checking_assert (csum);
    3615                 :     119283 :       if (csum->m_return_ignored)
    3616                 :      36608 :         continue;
    3617                 :      82675 :       if (!csum->m_return_returned)
    3618                 :            :         return true;
    3619                 :            : 
    3620                 :      15084 :       isra_func_summary *from_ifs = func_sums->get (cs->caller);
    3621                 :      15084 :       if (!from_ifs || !from_ifs->m_candidate)
    3622                 :       4509 :         return true;
    3623                 :            : 
    3624                 :      10575 :       if (!ipa_edge_within_scc (cs)
    3625                 :      10575 :           && !from_ifs->m_return_ignored)
    3626                 :            :             return true;
    3627                 :            :     }
    3628                 :            : 
    3629                 :            :   return false;
    3630                 :            : }
    3631                 :            : 
    3632                 :            : /* Push into NEW_PARAMS all required parameter adjustment entries to copy or
    3633                 :            :    modify parameter which originally had index BASE_INDEX, in the adjustment
    3634                 :            :    vector of parent clone (if any) had PREV_CLONE_INDEX and was described by
    3635                 :            :    PREV_ADJUSTMENT.  If the parent clone is the original function,
    3636                 :            :    PREV_ADJUSTMENT is NULL and PREV_CLONE_INDEX is equal to BASE_INDEX.  */
    3637                 :            : 
    3638                 :            : 
    3639                 :            : static void
    3640                 :     153977 : push_param_adjustments_for_index (isra_func_summary *ifs, unsigned base_index,
    3641                 :            :                                   unsigned prev_clone_index,
    3642                 :            :                                   ipa_adjusted_param *prev_adjustment,
    3643                 :            :                                   vec<ipa_adjusted_param, va_gc> **new_params)
    3644                 :            : {
    3645                 :     153977 :   isra_param_desc *desc = &(*ifs->m_parameters)[base_index];
    3646                 :     153977 :   if (desc->locally_unused)
    3647                 :            :     {
    3648                 :      30084 :       if (dump_file)
    3649                 :         23 :         fprintf (dump_file, "  Will remove parameter %u\n", base_index);
    3650                 :      30084 :       return;
    3651                 :            :     }
    3652                 :            : 
    3653                 :     123893 :   if (!desc->split_candidate)
    3654                 :            :     {
    3655                 :     107060 :       ipa_adjusted_param adj;
    3656                 :     107060 :       if (prev_adjustment)
    3657                 :            :         {
    3658                 :        609 :           adj = *prev_adjustment;
    3659                 :        609 :           adj.prev_clone_adjustment = true;
    3660                 :        609 :           adj.prev_clone_index = prev_clone_index;
    3661                 :            :         }
    3662                 :            :       else
    3663                 :            :         {
    3664                 :     106451 :           memset (&adj, 0, sizeof (adj));
    3665                 :     106451 :           adj.op = IPA_PARAM_OP_COPY;
    3666                 :     106451 :           adj.base_index = base_index;
    3667                 :     106451 :           adj.prev_clone_index = prev_clone_index;
    3668                 :            :         }
    3669                 :     107060 :       vec_safe_push ((*new_params), adj);
    3670                 :     107060 :       return;
    3671                 :            :     }
    3672                 :            : 
    3673                 :      16833 :   if (dump_file)
    3674                 :         16 :     fprintf (dump_file, "  Will split parameter %u\n", base_index);
    3675                 :            : 
    3676                 :      16833 :   gcc_assert (!prev_adjustment || prev_adjustment->op == IPA_PARAM_OP_COPY);
    3677                 :      16833 :   unsigned aclen = vec_safe_length (desc->accesses);
    3678                 :      46331 :   for (unsigned j = 0; j < aclen; j++)
    3679                 :            :     {
    3680                 :      29498 :       param_access *pa = (*desc->accesses)[j];
    3681                 :      29498 :       if (!pa->certain)
    3682                 :        730 :         continue;
    3683                 :      28768 :       if (dump_file)
    3684                 :         26 :         fprintf (dump_file, "    - component at byte offset %u, "
    3685                 :         26 :                  "size %u\n", pa->unit_offset, pa->unit_size);
    3686                 :            : 
    3687                 :      28768 :       ipa_adjusted_param adj;
    3688                 :      28768 :       memset (&adj, 0, sizeof (adj));
    3689                 :      28768 :       adj.op = IPA_PARAM_OP_SPLIT;
    3690                 :      28768 :       adj.base_index = base_index;
    3691                 :      28768 :       adj.prev_clone_index = prev_clone_index;
    3692                 :      28768 :       adj.param_prefix_index = IPA_PARAM_PREFIX_ISRA;
    3693                 :      28768 :       adj.reverse = pa->reverse;
    3694                 :      28768 :       adj.type = pa->type;
    3695                 :      28768 :       adj.alias_ptr_type = pa->alias_ptr_type;
    3696                 :      28768 :       adj.unit_offset = pa->unit_offset;
    3697                 :      28768 :       vec_safe_push ((*new_params), adj);
    3698                 :            :     }
    3699                 :            : }
    3700                 :            : 
    3701                 :            : /* Worker for all call_for_symbol_thunks_and_aliases.  Set calls_comdat_local
    3702                 :            :    flag of all callers of NODE.  */
    3703                 :            : 
    3704                 :            : static bool
    3705                 :          1 : mark_callers_calls_comdat_local (struct cgraph_node *node, void *)
    3706                 :            : {
    3707                 :          3 :   for (cgraph_edge *cs = node->callers; cs; cs = cs->next_caller)
    3708                 :          2 :     cs->caller->calls_comdat_local = true;
    3709                 :          1 :   return false;
    3710                 :            : }
    3711                 :            : 
    3712                 :            : 
    3713                 :            : /* Do final processing of results of IPA propagation regarding NODE, clone it
    3714                 :            :    if appropriate.  */
    3715                 :            : 
    3716                 :            : static void
    3717                 :     819260 : process_isra_node_results (cgraph_node *node,
    3718                 :            :                            hash_map<const char *, unsigned> *clone_num_suffixes)
    3719                 :            : {
    3720                 :     819260 :   isra_func_summary *ifs = func_sums->get (node);
    3721                 :     819260 :   if (!ifs || !ifs->m_candidate)
    3722                 :     619910 :     return;
    3723                 :            : 
    3724                 :     253506 :   auto_vec<bool, 16> surviving_params;
    3725                 :     199350 :   bool check_surviving = false;
    3726                 :     199350 :   if (node->clone.param_adjustments)
    3727                 :            :     {
    3728                 :      13015 :       check_surviving = true;
    3729                 :      13015 :       node->clone.param_adjustments->get_surviving_params (&surviving_params);
    3730                 :            :     }
    3731                 :            : 
    3732                 :     199350 :   unsigned param_count = vec_safe_length (ifs->m_parameters);
    3733                 :     199350 :   bool will_change_function = false;
    3734                 :     199350 :   if (ifs->m_returns_value && ifs->m_return_ignored)
    3735                 :            :     will_change_function = true;
    3736                 :            :   else
    3737                 :     480601 :     for (unsigned i = 0; i < param_count; i++)
    3738                 :            :       {
    3739                 :     335407 :         isra_param_desc *desc = &(*ifs->m_parameters)[i];
    3740                 :     335407 :         if ((desc->locally_unused || desc->split_candidate)
    3741                 :            :             /* Make sure we do not clone just to attempt to remove an already
    3742                 :            :                removed unused argument.  */
    3743                 :     335407 :             && (!check_surviving
    3744                 :       6374 :                 || (i < surviving_params.length ()
    3745                 :       1027 :                     && surviving_params[i])))
    3746                 :            :           {
    3747                 :            :             will_change_function = true;
    3748                 :            :             break;
    3749                 :            :           }
    3750                 :            :       }
    3751                 :     184053 :   if (!will_change_function)
    3752                 :     910298 :     return;
    3753                 :            : 
    3754                 :      54156 :   if (dump_file)
    3755                 :            :     {
    3756                 :         33 :       fprintf (dump_file, "\nEvaluating analysis results for %s\n",
    3757                 :            :                node->dump_name ());
    3758                 :         33 :       if (ifs->m_returns_value && ifs->m_return_ignored)
    3759                 :          9 :         fprintf (dump_file, "  Will remove return value.\n");
    3760                 :            :     }
    3761                 :            : 
    3762                 :      54156 :   vec<ipa_adjusted_param, va_gc> *new_params = NULL;
    3763                 :      54156 :   if (ipa_param_adjustments *old_adjustments = node->clone.param_adjustments)
    3764                 :            :     {
    3765                 :        747 :       unsigned old_adj_len = vec_safe_length (old_adjustments->m_adj_params);
    3766                 :       1567 :       for (unsigned i = 0; i < old_adj_len; i++)
    3767                 :            :         {
    3768                 :        820 :           ipa_adjusted_param *old_adj = &(*old_adjustments->m_adj_params)[i];
    3769                 :        820 :           push_param_adjustments_for_index (ifs, old_adj->base_index, i,
    3770                 :            :                                             old_adj, &new_params);
    3771                 :            :         }
    3772                 :            :     }
    3773                 :            :   else
    3774                 :     206566 :     for (unsigned i = 0; i < param_count; i++)
    3775                 :     153157 :       push_param_adjustments_for_index (ifs, i, i, NULL, &new_params);
    3776                 :            : 
    3777                 :      54156 :   ipa_param_adjustments *new_adjustments
    3778                 :            :     = (new (ggc_alloc <ipa_param_adjustments> ())
    3779                 :            :        ipa_param_adjustments (new_params, param_count,
    3780                 :      54156 :                               ifs->m_returns_value && ifs->m_return_ignored));
    3781                 :            : 
    3782                 :      54156 :   if (dump_file && (dump_flags & TDF_DETAILS))
    3783                 :            :     {
    3784                 :          1 :       fprintf (dump_file, "\n  Created adjustments:\n");
    3785                 :          1 :       new_adjustments->dump (dump_file);
    3786                 :            :     }
    3787                 :            : 
    3788                 :      54156 :   unsigned &suffix_counter = clone_num_suffixes->get_or_insert (
    3789                 :      54156 :                                IDENTIFIER_POINTER (DECL_ASSEMBLER_NAME (
    3790                 :      54156 :                                  node->decl)));
    3791                 :      54156 :   vec<cgraph_edge *> callers = node->collect_callers ();
    3792                 :      54156 :   cgraph_node *new_node
    3793                 :      54156 :     = node->create_virtual_clone (callers, NULL, new_adjustments, "isra",
    3794                 :            :                                   suffix_counter);
    3795                 :      54156 :   suffix_counter++;
    3796                 :      54156 :   if (node->calls_comdat_local && node->same_comdat_group)
    3797                 :            :     {
    3798                 :          1 :       new_node->add_to_same_comdat_group (node);
    3799                 :          1 :       new_node->call_for_symbol_and_aliases (mark_callers_calls_comdat_local,
    3800                 :            :                                              NULL, true);
    3801                 :            :     }
    3802                 :      54156 :   new_node->calls_comdat_local = node->calls_comdat_local;
    3803                 :            : 
    3804                 :      54156 :   if (dump_file)
    3805                 :         33 :     fprintf (dump_file, "  Created new node %s\n", new_node->dump_name ());
    3806                 :     108312 :   callers.release ();
    3807                 :            : }
    3808                 :            : 
    3809                 :            : /* Check which parameters of NODE described by IFS have survived until IPA-SRA
    3810                 :            :    and disable transformations for those which have not or which should not
    3811                 :            :    transformed because the associated debug counter reached its limit.  Return
    3812                 :            :    true if none survived or if there were no candidates to begin with.  */
    3813                 :            : 
    3814                 :            : static bool
    3815                 :     199350 : disable_unavailable_parameters (cgraph_node *node, isra_func_summary *ifs)
    3816                 :            : {
    3817                 :     199350 :   bool ret = true;
    3818                 :     199350 :   unsigned len = vec_safe_length (ifs->m_parameters);
    3819                 :     172124 :   if (!len)
    3820                 :            :     return true;
    3821                 :            : 
    3822                 :     172124 :   auto_vec<bool, 16> surviving_params;
    3823                 :     172124 :   bool check_surviving = false;
    3824                 :     172124 :   if (node->clone.param_adjustments)
    3825                 :            :     {
    3826                 :      13015 :       check_surviving = true;
    3827                 :      13015 :       node->clone.param_adjustments->get_surviving_params (&surviving_params);
    3828                 :            :     }
    3829                 :     172124 :   bool dumped_first = false;
    3830                 :     576210 :   for (unsigned i = 0; i < len; i++)
    3831                 :            :     {
    3832                 :     404086 :       isra_param_desc *desc = &(*ifs->m_parameters)[i];
    3833                 :     404086 :       if (!dbg_cnt (ipa_sra_params))
    3834                 :            :         {
    3835                 :          0 :           desc->locally_unused = false;
    3836                 :          0 :           desc->split_candidate = false;
    3837                 :            :         }
    3838                 :     404086 :       else if (check_surviving
    3839                 :     435456 :                && (i >= surviving_params.length ()
    3840                 :      10422 :                    || !surviving_params[i]))
    3841                 :            :         {
    3842                 :            :           /* Even if the parameter was removed by a previous IPA pass, we do
    3843                 :            :              not clear locally_unused because if it really is unused, this
    3844                 :            :              information might be useful in callers.  */
    3845                 :      23233 :           desc->split_candidate = false;
    3846                 :            : 
    3847                 :      23233 :           if (dump_file && (dump_flags & TDF_DETAILS))
    3848                 :            :             {
    3849                 :          0 :               if (!dumped_first)
    3850                 :            :                 {
    3851                 :          0 :                   fprintf (dump_file,
    3852                 :            :                            "The following parameters of %s are dead on "
    3853                 :            :                            "arrival:", node->dump_name ());
    3854                 :          0 :                   dumped_first = true;
    3855                 :            :                 }
    3856                 :          0 :               fprintf (dump_file, " %u", i);
    3857                 :            :             }
    3858                 :            :         }
    3859                 :     380853 :       else if (desc->locally_unused || desc->split_candidate)
    3860                 :     133317 :         ret = false;
    3861                 :            :     }
    3862                 :            : 
    3863                 :     172124 :   if (dumped_first)
    3864                 :          0 :     fprintf (dump_file, "\n");
    3865                 :            : 
    3866                 :     172124 :   return ret;
    3867                 :            : }
    3868                 :            : 
    3869                 :            : 
    3870                 :            : /* Run the interprocedural part of IPA-SRA. */
    3871                 :            : 
    3872                 :            : static unsigned int
    3873                 :      93364 : ipa_sra_analysis (void)
    3874                 :            : {
    3875                 :      93364 :   if (dump_file)
    3876                 :            :     {
    3877                 :         35 :       fprintf (dump_file, "\n========== IPA-SRA IPA stage ==========\n");
    3878                 :         35 :       ipa_sra_dump_all_summaries (dump_file);
    3879                 :            :     }
    3880                 :            : 
    3881                 :      93364 :   gcc_checking_assert (func_sums);
    3882                 :      93364 :   gcc_checking_assert (call_sums);
    3883                 :      93364 :   cgraph_node **order = XCNEWVEC (cgraph_node *, symtab->cgraph_count);
    3884                 :      93364 :   auto_vec <cgraph_node *, 16> stack;
    3885                 :      93364 :   int node_scc_count = ipa_reduced_postorder (order, true, NULL);
    3886                 :            : 
    3887                 :            :   /* One sweep from callees to callers for parameter removal and splitting.  */
    3888                 :     958770 :   for (int i = 0; i < node_scc_count; i++)
    3889                 :            :     {
    3890                 :     865406 :       cgraph_node *scc_rep = order[i];
    3891                 :     865406 :       vec<cgraph_node *> cycle_nodes = ipa_get_nodes_in_cycle (scc_rep);
    3892                 :     865406 :       unsigned j;
    3893                 :            : 
    3894                 :            :       /* Preliminary IPA function level checks and first step of parameter
    3895                 :            :          removal.  */
    3896                 :     865406 :       cgraph_node *v;
    3897                 :    1736050 :       FOR_EACH_VEC_ELT (cycle_nodes, j, v)
    3898                 :            :         {
    3899                 :     870648 :           isra_func_summary *ifs = func_sums->get (v);
    3900                 :     870648 :           if (!ifs || !ifs->m_candidate)
    3901                 :     274237 :             continue;
    3902                 :     596411 :           if (!ipa_sra_ipa_function_checks (v)
    3903                 :     596411 :               || check_all_callers_for_issues (v))
    3904                 :            :             {
    3905                 :     397061 :               ifs->zap ();
    3906                 :     397061 :               continue;
    3907                 :            :             }
    3908                 :     199350 :           if (disable_unavailable_parameters (v, ifs))
    3909                 :     114993 :             continue;
    3910                 :      93077 :           for (cgraph_edge *cs = v->indirect_calls; cs; cs = cs->next_callee)
    3911                 :       8720 :             process_edge_to_unknown_caller (cs);
    3912                 :     393983 :           for (cgraph_edge *cs = v->callees; cs; cs = cs->next_callee)
    3913                 :     309626 :             if (!ipa_edge_within_scc (cs))
    3914                 :     302724 :               param_removal_cross_scc_edge (cs);
    3915                 :            :         }
    3916                 :            : 
    3917                 :            :       /* Look at edges within the current SCC and propagate used-ness across
    3918                 :            :          them, pushing onto the stack all notes which might need to be
    3919                 :            :          revisited.  */
    3920                 :    1736050 :       FOR_EACH_VEC_ELT (cycle_nodes, j, v)
    3921                 :     870648 :         v->call_for_symbol_thunks_and_aliases (propagate_used_to_scc_callers,
    3922                 :            :                                                &stack, true);
    3923                 :            : 
    3924                 :            :       /* Keep revisiting and pushing until nothing changes.  */
    3925                 :     867254 :       while (!stack.is_empty ())
    3926                 :            :         {
    3927                 :       1848 :           cgraph_node *v = stack.pop ();
    3928                 :       1848 :           isra_func_summary *ifs = func_sums->get (v);
    3929                 :       1848 :           gcc_checking_assert (ifs && ifs->m_queued);
    3930                 :       1848 :           ifs->m_queued = false;
    3931                 :            : 
    3932                 :       1848 :           v->call_for_symbol_thunks_and_aliases (propagate_used_to_scc_callers,
    3933                 :            :                                                  &stack, true);
    3934                 :            :         }
    3935                 :            : 
    3936                 :            :       /* Parameter splitting.  */
    3937                 :     883908 :       bool repeat_scc_access_propagation;
    3938                 :    1767820 :       do
    3939                 :            :         {
    3940                 :     883908 :           repeat_scc_access_propagation = false;
    3941                 :    2660000 :           FOR_EACH_VEC_ELT (cycle_nodes, j, v)
    3942                 :            :             {
    3943                 :     892183 :               isra_func_summary *ifs = func_sums->get (v);
    3944                 :     255868 :               if (!ifs
    3945                 :     636315 :                   || !ifs->m_candidate
    3946                 :     856945 :                   || vec_safe_is_empty (ifs->m_parameters))
    3947                 :     698779 :                 continue;
    3948                 :     877662 :               for (cgraph_edge *cs = v->callees; cs; cs = cs->next_callee)
    3949                 :     684258 :                 if (param_splitting_across_edge (cs))
    3950                 :      20576 :                   repeat_scc_access_propagation = true;
    3951                 :            :             }
    3952                 :            :         }
    3953                 :            :       while (repeat_scc_access_propagation);
    3954                 :            : 
    3955                 :     865406 :       if (flag_checking)
    3956                 :    1736040 :         FOR_EACH_VEC_ELT (cycle_nodes, j, v)
    3957                 :     870643 :           verify_splitting_accesses (v, true);
    3958                 :            : 
    3959                 :    1730810 :       cycle_nodes.release ();
    3960                 :            :     }
    3961                 :            : 
    3962                 :            :   /* One sweep from caller to callees for result removal.  */
    3963                 :     958770 :   for (int i = node_scc_count - 1; i >= 0 ; i--)
    3964                 :            :     {
    3965                 :     865406 :       cgraph_node *scc_rep = order[i];
    3966                 :     865406 :       vec<cgraph_node *> cycle_nodes = ipa_get_nodes_in_cycle (scc_rep);
    3967                 :     865406 :       unsigned j;
    3968                 :            : 
    3969                 :     865406 :       cgraph_node *v;
    3970                 :    1736050 :       FOR_EACH_VEC_ELT (cycle_nodes, j, v)
    3971                 :            :         {
    3972                 :     870648 :           isra_func_summary *ifs = func_sums->get (v);
    3973                 :     870648 :           if (!ifs || !ifs->m_candidate)
    3974                 :     671298 :             continue;
    3975                 :            : 
    3976                 :     199350 :           bool return_needed
    3977                 :            :             = (ifs->m_returns_value
    3978                 :      95370 :                && (!dbg_cnt (ipa_sra_retvalues)
    3979                 :      95370 :                    || v->call_for_symbol_and_aliases (retval_used_p,
    3980                 :     199350 :                                                       NULL, true)));
    3981                 :     199350 :           ifs->m_return_ignored = !return_needed;
    3982                 :     199350 :           if (return_needed)
    3983                 :     159680 :             isra_push_node_to_stack (v, ifs, &stack);
    3984                 :            :         }
    3985                 :            : 
    3986                 :     945479 :       while (!stack.is_empty ())
    3987                 :            :         {
    3988                 :      80073 :           cgraph_node *node = stack.pop ();
    3989                 :      80073 :           isra_func_summary *ifs = func_sums->get (node);
    3990                 :      80073 :           gcc_checking_assert (ifs && ifs->m_queued);
    3991                 :      80073 :           ifs->m_queued = false;
    3992                 :            : 
    3993                 :     405809 :           for (cgraph_edge *cs = node->callees; cs; cs = cs->next_callee)
    3994                 :     325736 :             if (ipa_edge_within_scc (cs)
    3995                 :     329146 :                 && call_sums->get (cs)->m_return_returned)
    3996                 :            :               {
    3997                 :        643 :                 enum availability av;
    3998                 :        643 :                 cgraph_node *callee = cs->callee->function_symbol (&av);
    3999                 :        643 :                 isra_func_summary *to_ifs = func_sums->get (callee);
    4000                 :        643 :                 if (to_ifs && to_ifs->m_return_ignored)
    4001                 :            :                   {
    4002                 :        233 :                     to_ifs->m_return_ignored = false;
    4003                 :        466 :                     isra_push_node_to_stack (callee, to_ifs, &stack);
    4004                 :            :                   }
    4005                 :            :               }
    4006                 :            :         }
    4007                 :    1730810 :       cycle_nodes.release ();
    4008                 :            :     }
    4009                 :            : 
    4010                 :      93364 :   ipa_free_postorder_info ();
    4011                 :      93364 :   free (order);
    4012                 :            : 
    4013                 :      93364 :   if (dump_file)
    4014                 :            :     {
    4015                 :         35 :       if (dump_flags & TDF_DETAILS)
    4016                 :            :         {
    4017                 :          2 :           fprintf (dump_file, "\n========== IPA-SRA propagation final state "
    4018                 :            :                    " ==========\n");
    4019                 :          2 :           ipa_sra_dump_all_summaries (dump_file);
    4020                 :            :         }
    4021                 :         35 :       fprintf (dump_file, "\n========== IPA-SRA decisions ==========\n");
    4022                 :            :     }
    4023                 :            : 
    4024                 :      93364 :   hash_map<const char *, unsigned> *clone_num_suffixes
    4025                 :      93364 :     = new hash_map<const char *, unsigned>;
    4026                 :            : 
    4027                 :      93364 :   cgraph_node *node;
    4028                 :    1825250 :   FOR_EACH_FUNCTION_WITH_GIMPLE_BODY (node)
    4029                 :     819260 :     process_isra_node_results (node, clone_num_suffixes);
    4030                 :            : 
    4031                 :      93364 :   delete clone_num_suffixes;
    4032                 :      93364 :   ggc_delete (func_sums);
    4033                 :      93364 :   func_sums = NULL;
    4034                 :      93364 :   delete call_sums;
    4035                 :      93364 :   call_sums = NULL;
    4036                 :            : 
    4037                 :      93364 :   if (dump_file)
    4038                 :         35 :     fprintf (dump_file, "\n========== IPA SRA IPA analysis done "
    4039                 :            :              "==========\n\n");
    4040                 :      93364 :   return 0;
    4041                 :            : }
    4042                 :            : 
    4043                 :            : 
    4044                 :            : const pass_data pass_data_ipa_sra =
    4045                 :            : {
    4046                 :            :   IPA_PASS, /* type */
    4047                 :            :   "sra", /* name */
    4048                 :            :   OPTGROUP_NONE, /* optinfo_flags */
    4049                 :            :   TV_IPA_SRA, /* tv_id */
    4050                 :            :   0, /* properties_required */
    4051                 :            :   0, /* properties_provided */
    4052                 :            :   0, /* properties_destroyed */
    4053                 :            :   0, /* todo_flags_start */
    4054                 :            :   ( TODO_dump_symtab | TODO_remove_functions ), /* todo_flags_finish */
    4055                 :            : };
    4056                 :            : 
    4057                 :            : class pass_ipa_sra : public ipa_opt_pass_d
    4058                 :            : {
    4059                 :            : public:
    4060                 :     200773 :   pass_ipa_sra (gcc::context *ctxt)
    4061                 :            :     : ipa_opt_pass_d (pass_data_ipa_sra, ctxt,
    4062                 :            :                       ipa_sra_generate_summary, /* generate_summary */
    4063                 :            :                       ipa_sra_write_summary, /* write_summary */
    4064                 :            :                       ipa_sra_read_summary, /* read_summary */
    4065                 :            :                       NULL , /* write_optimization_summary */
    4066                 :            :                       NULL, /* read_optimization_summary */
    4067                 :            :                       NULL, /* stmt_fixup */
    4068                 :            :                       0, /* function_transform_todo_flags_start */
    4069                 :            :                       NULL, /* function_transform */
    4070                 :     401546 :                       NULL) /* variable_transform */
    4071                 :            :   {}
    4072                 :            : 
    4073                 :            :   /* opt_pass methods: */
    4074                 :     432166 :   virtual bool gate (function *)
    4075                 :            :     {
    4076                 :            :       /* TODO: We should remove the optimize check after we ensure we never run
    4077                 :            :          IPA passes when not optimizing.  */
    4078                 :     432166 :       return (flag_ipa_sra && optimize);
    4079                 :            :     }
    4080                 :            : 
    4081                 :      93364 :   virtual unsigned int execute (function *) { return ipa_sra_analysis (); }
    4082                 :            : 
    4083                 :            : }; // class pass_ipa_sra
    4084                 :            : 
    4085                 :            : } // anon namespace
    4086                 :            : 
    4087                 :            : ipa_opt_pass_d *
    4088                 :     200773 : make_pass_ipa_sra (gcc::context *ctxt)
    4089                 :            : {
    4090                 :     200773 :   return new pass_ipa_sra (ctxt);
    4091                 :            : }
    4092                 :            : 
    4093                 :            : 
    4094                 :            : #include "gt-ipa-sra.h"

Generated by: LCOV version 1.0

LCOV profile is generated on x86_64 machine using following configure options: configure --disable-bootstrap --enable-coverage=opt --enable-languages=c,c++,fortran,go,jit,lto --enable-host-shared. GCC test suite is run with the built compiler.