Managing Compiler Errata

---

Abstract

Ensur­ing the reli­a­bil­i­ty and safe­ty of func­tion­al safe­ty-relat­ed soft­ware neces­si­tates effec­tive man­age­ment of com­pil­er erra­ta. Com­pil­er issues can sig­nif­i­cant­ly com­pro­mise soft­ware integri­ty, mak­ing it essen­tial to mon­i­tor and mit­i­gate these risks through­out the prod­uct life­cy­cle. This arti­cle explores var­i­ous strate­gies and tools for man­ag­ing com­pil­er errors, eval­u­at­ing their effec­tive­ness in terms of qual­i­ty-of-results (QoR) and asso­ci­at­ed costs.

Problem definition

Although gen­er­al­ly reli­able, com­pil­ers are not with­out flaws. Issues typ­i­cal­ly emerge in cor­ner cas­es and may not always affect the behav­ior of the com­piled soft­ware. How­ev­er, for soft­ware that must com­ply with func­tion­al safe­ty stan­dards, it is essen­tial to demon­strate that the soft­ware is unaf­fect­ed by these issues or to update the soft­ware accord­ing­ly. When soft­ware is devel­oped by mul­ti­ple par­ties and deployed across numer­ous devices, updates can be extreme­ly cost­ly. There­fore, con­struct­ing a com­pelling argu­ment that the soft­ware remains unaf­fect­ed by com­pil­er issues can be a more eco­nom­i­cal solu­tion.

Managing compiler errata

The first step in man­ag­ing com­pil­er erra­ta is to mit­i­gate the risk of being affect­ed by com­pil­er issues. This can be achieved by adher­ing to meth­ods out­lined in func­tion­al safe­ty stan­dards such as ISO 26262, DO-178C, and IEC 61508. Key strate­gies include: using a cer­ti­fied com­pil­er and ensur­ing that your use case is com­pat­i­ble with a qual­i­fied use case, fol­low­ing cod­ing guide­lines such as MISRA or CERT C and uti­liz­ing sta­t­ic analy­sis tools, and con­sid­er­ing poten­tial com­pil­er errors dur­ing safe­ty and secu­ri­ty analy­sis.

In addi­tion to apply­ing these risk reduc­tion mea­sures, it is impor­tant to track com­pil­er erra­ta also after a prod­uct has been released. Reg­u­lar­ly review the com­pil­er’s pub­lished erra­ta or known issue list and apply ven­dor-rec­om­mend­ed mit­i­ga­tions, which can include com­pil­er-spe­cif­ic workarounds or updates. Once a prod­uct is in the field, close col­lab­o­ra­tion with the com­pil­er ven­dor is nec­es­sary for sup­port and insights on whether erra­ta affect your soft­ware.

Compiler errata databases

All com­mer­cial and open-source com­pil­er sup­pli­ers track issues using a data­base. For an exam­ple GCC Bugzil­la is a resource where you can find bug reports. If you take a clos­er look you may con­clude that track­ing such a large num­ber of issues can be chal­leng­ing for orga­ni­za­tions.

Com­pil­er ven­dors tar­get­ing the FuSa domain ana­lyze and clas­si­fy known issues based on the rel­a­tive risk they pose to the com­piled soft­ware’s behav­ior. Only issues that could cause mis­be­hav­ior in the com­piled soft­ware need to be tracked.

A typ­i­cal bug report includes a descrip­tion of the con­di­tions under which the bug man­i­fest­ed and a ref­er­ence to the com­pil­er ver­sion used. Once the com­pil­er sup­pli­er has ana­lyzed the bug addi­tion­al infor­ma­tion is added such as, the cur­rent sta­tus of the bug, the ver­sions of the com­pil­er that are affect­ed by the bug, ways to mit­i­gate or work around the issue, and the ver­sion in which the bug is solved.

To help in under­stand­ing the nature and impact of com­pil­er bugs, they can be clas­si­fied into sev­er­al cat­e­gories:

• Mis­com­pi­la­tion Bugs occur when the com­pil­er gen­er­ates incor­rect code from valid input. This can be due to flawed opti­miza­tions, inad­e­quate safe­ty checks, or unsound sta­t­ic analy­ses.
• Crash Bugs cause the com­pil­er to crash dur­ing the com­pi­la­tion process.
• Silent Bugs intro­duce errors in the com­piled code that are dif­fi­cult to detect because they don’t cause imme­di­ate fail­ures. Exam­ples include con­cur­ren­cy bugs and mem­o­ry leaks.
• Workaround Bugs are bugs that pro­gram­mers can bypass using tem­po­rary fix­es or workarounds until the bugs are offi­cial­ly resolved.

This arti­cle focus­es on mis­com­pi­la­tion and silent bugs caused by code gen­er­a­tion, opti­miza­tion, and link­ing errors, as these pose a high risk of caus­ing the com­piled soft­ware to mal­func­tion. Exam­ples include reg­is­ter allo­ca­tion bugs, incor­rect loop trans­for­ma­tions, faulty intrapro­ce­dur­al opti­miza­tions, and erro­neous dead code elim­i­na­tion.

The chal­lenge with these bugs is that it’s often impos­si­ble to iden­ti­fy all soft­ware con­structs that could trig­ger them from the issue descrip­tion alone. While cod­ing stan­dards and sta­t­ic analy­sis tools aim to keep source code com­plex­i­ty low, com­plex­i­ty increas­es sig­nif­i­cant­ly at the inter­me­di­ate lan­guage lev­el used inter­nal­ly by com­pil­ers once opti­miza­tions are applied. Addi­tion­al­ly, a bug in a spe­cif­ic opti­miza­tion phase may have its root cause in an analy­sis phase, such as alias analy­sis, which is used by oth­er opti­miza­tion phas­es. There­fore, even for the com­pil­er engi­neer fix­ing the bug, it can be impos­si­ble to describe all poten­tial source code con­structs trig­ger­ing the bug. This under­ly­ing com­plex­i­ty neg­a­tive­ly impacts the qual­i­ty of results of most meth­ods for ver­i­fy­ing whether soft­ware is affect­ed by a com­pil­er bug.

Methods for verifying whether software is affected by compiler issues

In the auto­mo­tive indus­try, var­i­ous meth­ods are employed to sys­tem­at­i­cal­ly detect whether soft­ware is affect­ed by known com­pil­er bugs. The ini­tial step involves under­stand­ing the erra­ta by exam­in­ing their descrip­tions, which include the affect­ed com­pil­er ver­sions and spe­cif­ic soft­ware con­structs or pat­terns that trig­ger the bug. This assess­ment helps deter­mine if the erra­ta apply to your project.

To per­form this activ­i­ty effec­tive­ly, a cer­tain lev­el of exper­tise in the C lan­guage, the tar­get archi­tec­ture, and Func­tion­al Safe­ty (FuSa) aspects of the user soft­ware is required. The effi­cien­cy and effec­tive­ness of this activ­i­ty large­ly depend on the qual­i­ty of the bug report pro­vid­ed by the com­pil­er sup­pli­er.

Often, a sta­t­ic analy­sis tool is used to iden­ti­fy con­structs in the source code that match pat­terns described in the erra­ta. If the erra­ta describe a par­tic­u­lar source code con­struct, grep-like tools or more advanced equiv­a­lence check­ers can be used to locate occur­rences of prob­lem­at­ic con­structs in the code­base. How­ev­er, the results of this activ­i­ty are often dis­ap­point­ing because the issue descrip­tion typ­i­cal­ly shows one spe­cif­ic code frag­ment caus­ing the issue. What is need­ed is a gener­ic descrip­tion of the root cause of the issue and how this root cause prop­a­gates to the C source lan­guage lev­el. Such detailed issue descrip­tions are rarely pro­vid­ed in bug reports and are often dif­fi­cult to devel­op for the class of bugs addressed in this arti­cle.

If the com­pil­er ven­dor pro­vides a spe­cif­ic diag­nos­tic or mit­i­ga­tion for an issue, it is rel­a­tive­ly straight­for­ward to apply these solu­tions dur­ing the soft­ware develop­ment phase. How­ev­er, once the soft­ware has been released, it is gen­er­al­ly not fea­si­ble to use the mit­i­ga­tion to deter­mine if the soft­ware is affect­ed by the issue. Only when the effects of a mit­i­ga­tion are fine-grained can a sub­se­quent bina­ry inspec­tion of the assem­bly or bina­ry file be suc­cess­ful­ly per­formed in prac­tice.

Code instru­men­ta­tion, which involves insert­ing log­ging or asser­tions around areas sus­pect­ed of being affect­ed by the erra­ta, presents sev­er­al chal­lenges. Often, the issue descrip­tion lacks suf­fi­cient detail to iden­ti­fy all the areas that require instru­men­ta­tion. Addi­tion­al­ly, this method is intru­sive and can adverse­ly affect the soft­ware’s tim­ing behav­ior, lead­ing to low-qual­i­ty out­comes and high costs.

For­mal ver­i­fi­ca­tion of whether the seman­tics of the source code match the instruc­tions in the bina­ry file is fea­si­ble today, but with the cur­rent state of the tech­nol­o­gy, it can only be suc­cess­ful­ly applied to small code bases. For large code bases, one must lim­it one­self to ver­i­fy­ing safe­ty-crit­i­cal code sec­tions only.

Com­pil­er ven­dors tar­get­ing the FuSa domain may offer bug-check­ing com­pil­ers. These tools func­tion like stan­dard com­pil­ers but gen­er­ate a diag­nos­tic mes­sage pin­point­ing the loca­tions in the user’s source code that trig­ger the com­pil­er bug. Typ­i­cal­ly, these bug-check­ing com­pil­ers are avail­able only upon cus­tomer request and to those who have this ser­vice includ­ed in their sup­port con­tract. How­ev­er, some ven­dors like TASKING offer bug-check­ing com­pil­ers as sup­ple­men­tal prod­ucts to their stan­dard com­pil­ers. These com­pil­er ven­dor-spe­cif­ic tools typ­i­cal­ly pro­vide the high­est qual­i­ty results.

FuSa stan­dards pro­vide detailed, safe­ty lev­el-depen­dent require­ments for ver­i­fi­ca­tion and test process­es. How­ev­er, dur­ing the ini­tial develop­ment of the soft­ware, the ver­i­fi­ca­tion activ­i­ties did not reveal any prob­lems caused by com­pil­er issues that were unknown at that time but emerged after the soft­ware’s release. This rais­es the ques­tion: were the applied ver­i­fi­ca­tion activ­i­ties insuf­fi­cient, or is the soft­ware gen­uine­ly unaf­fect­ed by the new­ly iden­ti­fied com­pil­er issue? To answer this ques­tion, spe­cial atten­tion must be giv­en to tests that exer­cise the con­structs men­tioned in the erra­ta. If there are doubts about the com­plete­ness of the test set, spe­cif­ic test cas­es mim­ic­k­ing the sce­nar­ios described in the erra­ta could be devel­oped, and the alter­na­tive meth­ods out­lined in this sec­tion should be applied.

In the remain­der of this arti­cle, we explore using exist­ing ASIL-com­pli­ant test sets in com­bi­na­tion with a bug-check­ing com­pil­er to ver­i­fy whether soft­ware is affect­ed by com­pil­er issues.

Using a Bug-checking compiler (Inspector tool)

A bug-check­ing com­pil­er is a spe­cial­ized type of com­pil­er designed to iden­ti­fy and diag­nose bugs in soft­ware. These tools exe­cute quick­ly, mak­ing them suit­able for ana­lyz­ing large code bases. Unlike stan­dard com­pil­ers, which trans­late code from a high-lev­el pro­gram­ming lan­guage to machine code, bug-check­ing com­pil­ers include detec­tors that iden­ti­fy code con­structs in the user’s soft­ware that trig­ger known com­pil­er bugs. This helps devel­op­ers quick­ly under­stand and address issues. The large­ly auto­mat­ed analy­sis phase reduces the need for man­u­al inspec­tion, sig­nif­i­cant­ly increas­ing the effi­cien­cy of iden­ti­fy­ing bugs.

TASKING pro­vides a bug-check­ing com­pil­er called the “Inspec­tor tool” along­side their stan­dard com­pil­ers. This tool is capa­ble of iden­ti­fy­ing most known issues. Below is an excerpt from the TASKING Issue Por­tal, which lists known bugs and indi­cates whether a detec­tor is avail­able in the Inspec­tor tool to ana­lyze if the user’s soft­ware is affect­ed by the bug. There are lim­i­ta­tions on whether an inspec­tor is pro­vid­ed. Issues not in scope include SIL‑1 issues, as they do not pose a FuSa risk, issues not detectable sta­t­i­cal­ly at com­pile time, and library issues, as most are iden­ti­fi­able only at exe­cu­tion time. Gen­er­al­ly speak­ing, for most mis­com­pi­la­tion and silent bugs, a detec­tor is avail­able.

Inte­grat­ing a bug-check­ing com­pil­er into the soft­ware develop­ment work­flow is straight­for­ward. For TASKING’s Inspec­tor tool, this involves con­fig­ur­ing the build sys­tem to use the Inspec­tor instead of the stan­dard com­pil­er. Both tools sup­port the same com­mand line options, with addi­tion­al options in the Inspec­tor to enable or dis­able spe­cif­ic detec­tors for which fix­es have already been pro­vid­ed. Ide­al­ly the bug-check­ing com­pil­er is inte­grat­ed into the con­tin­u­ous inte­gra­tion (CI) pipeline ensur­ing that every code change is auto­mat­i­cal­ly checked for known com­pil­er bugs, main­tain­ing soft­ware integri­ty through­out its life­cy­cle.

Diag­nos­tics pro­vid­ed by a bug-check­ing com­pil­er can be clas­si­fied into two cat­e­gories:

• Def­i­nite Impact: The user soft­ware, at the point described in the diag­nos­tic, is def­i­nite­ly affect­ed by the com­pil­er erra­ta, with no false pos­i­tives.
• Poten­tial Impact: The user soft­ware, at the point described in the diag­nos­tic, is poten­tial­ly affect­ed by the com­pil­er erra­ta, with pos­si­ble false pos­i­tives.

Review­ing diag­nos­tic mes­sages is cru­cial.

Deal­ing with def­i­nite diag­nos­tics is straight­for­ward. The user is informed about the exact loca­tion in the soft­ware where the bug impacts the cor­rect­ness of the gen­er­at­ed com­pil­er out­put. The mit­i­ga­tion field of the erra­ta describes the actions to mit­i­gate the effects of the bug. Based on this infor­ma­tion, the user must assess whether the impli­ca­tions of the bug neces­si­tate an update to the soft­ware.

Deal­ing with poten­tial detec­tions can be more chal­leng­ing, depend­ing on the num­ber of false pos­i­tives. If the num­ber is low, each diag­nos­tic mes­sage can be assessed indi­vid­u­al­ly. If the num­ber is high, this may become trou­ble­some. How­ev­er, the bug-check­ing com­pil­er is the most state-of-the-art tool for detect­ing whether soft­ware is affect­ed by a com­pil­er bug. There­fore, the best way for­ward is to first demon­strate that the num­ber of false pos­i­tives is high to gain a lev­el of con­fi­dence that the soft­ware is unaf­fect­ed by the com­pil­er bug, and com­bine that evi­dence with the results of the FuSa stan­dard ASIL-com­pli­ant ver­i­fi­ca­tion and tests require­ments that apply to the soft­ware.

After mod­i­fy­ing the soft­ware, con­duct FuSa stan­dard-com­pli­ant ver­i­fi­ca­tion and test­ing to ensure the fix­es have resolved the iden­ti­fied issues.

State of the Art – Quality Assurance

TASKING’s Inspec­tor tool is devel­oped by the same engi­neer­ing team that cre­at­ed the com­pil­er, using the same process­es. This ensures a deep under­stand­ing of the com­pil­er’s design and adher­ence to process matu­ri­ty stan­dards such as ASPICE.

The imple­men­ta­tion of a bug-check­ing com­pil­er mir­rors that of the asso­ci­at­ed com­pil­er, with addi­tion­al code – called detec­tors – to issue diag­nos­tic mes­sages when the root cause of a bug is detect­ed. These detec­tors must be non-intru­sive and shall not alter the inter­nal state of the com­pil­er as this would com­pro­mise the reli­a­bil­i­ty of the tool.

To ver­i­fy non-intru­sive­ness, the bug-check­ing com­pil­er inher­its the test suites of the com­pil­er. Under test con­di­tions, the bug-check­ing com­pil­er can gen­er­ate bina­ry code. The bina­ry code gen­er­at­ed by the bug-check­ing com­pil­er should be iden­ti­cal to the code gen­er­at­ed by the asso­ci­at­ed com­pil­er. This test ensures that the diag­nos­tics imple­ment­ed in the bug-check­ing com­pil­er do not affect its inter­nal state.

Per detec­tor, addi­tion­al tests are devel­oped to ver­i­fy that diag­nos­tic mes­sages are issued when required.

Addi­tion­al­ly, the Inspec­tor tool is used to ver­i­fy whether the libraries that are shipped with the com­pil­er are impact­ed by a com­pil­er bug.

Tool Confidence Level Argument

The Inspec­tor tool is used in projects that must com­ply with safe­ty stan­dards, such as ISO 26262. The asso­ci­at­ed tool qual­i­fi­ca­tion process is depict­ed in Fig­ure 1.

The Tool Impact lev­el (TI) of a bug-check­ing com­pil­er is TI2, as there is a pos­si­bil­i­ty that a mal­func­tion of the tool could fail to detect errors in a safe­ty-relat­ed item or ele­ment.

When select­ing the appro­pri­ate Tool Error Detec­tion (TD) lev­el, the user’s ver­i­fi­ca­tion process must be con­sid­ered. Accord­ing to ISO 26262, TD1 should be select­ed if there is a high degree of con­fi­dence that a mal­func­tion and its cor­re­spond­ing out­puts will be pre­vent­ed or detect­ed. Since the tool is typ­i­cal­ly deployed in projects with high ASILs, the user’s develop­ment and ver­i­fi­ca­tion process­es com­ply with the strin­gent require­ments of such projects, ensur­ing a high lev­el of con­fi­dence that errors are detect­ed by the default ver­i­fi­ca­tion process. The Inspec­tor tool is used to com­ple­ment the user’s ver­i­fi­ca­tion process, there­fore TD1 applies.

As a con­se­quence, the Tool Con­fi­dence Lev­el is TCL1, mean­ing that no qual­i­fi­ca­tion meth­ods are required.

Conclusion

Man­ag­ing com­pil­er erra­ta is essen­tial for ensur­ing the reli­a­bil­i­ty and safe­ty of func­tion­al safe­ty-relat­ed soft­ware through­out its life­cy­cle. Advanced bug-check­ing tools, such as TASK­ING’s Inspec­tor, offer state-of-the-art solu­tions for detect­ing whether com­pil­er bugs com­pro­mise the integri­ty of the user’s soft­ware. Since bugs can be dis­cov­ered at any time, it is cru­cial that these tools are con­tin­u­ous­ly updat­ed.

By uti­liz­ing tools like Inspec­tor, the erra­ta man­age­ment process becomes large­ly auto­mat­ed, sig­nif­i­cant­ly reduc­ing the need for man­u­al inter­ven­tion and lead­ing to sub­stan­tial cost sav­ings.

Authors

Peter Him­ler, Ger­ard Vink, both part of TASKING’s Busi­ness Develop­ment and Inno­va­tion team.

Ger­man ver­sion avail­able here.