You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-20 - 12:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Thu Aug 20, 2026 00:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13249521870,5sleep30-21swapper/319:47:453
28012621190,3chrt0-21swapper/322:17:453
2206842770,5chrt0-21swapper/221:17:452
92909997464,6cyclictest382843-21turbostat.cron00:00:001
2754312710,7chrt291ktimers/122:12:461
92909997062,6cyclictest382828-21kworker/u8:3+flush-8:000:08:041
92909996660,4cyclictest119450-21kworker/u8:0+flush-8:019:45:011
1487432660,5chrt0-21swapper/120:02:481
92907996252,7cyclictest298552-21latency_hist22:35:010
92907995953,4cyclictest412884-21latency_hist00:30:010
92909995751,3cyclictest164253-21kworker/u8:2+flush-8:020:27:561
92909995747,7cyclictest398040-21kworker/u8:1+flush-8:000:22:501
92907995747,7cyclictest248835-21latency_hist21:45:010
92909995645,8cyclictest119450-21kworker/u8:0+flush-8:019:57:511
1866622560,4sleep1281rcuc/120:42:451
92909995447,5cyclictest248798-21kworker/u8:0+flush-8:021:52:481
92909995444,7cyclictest184416-21kworker/u8:1+flush-8:022:02:531
92909995444,7cyclictest184416-21kworker/u8:1+flush-8:022:02:531
92907995445,6cyclictest92757-21latency_hist19:10:010
92909995346,5cyclictest372681-21kworker/u8:0+flush-8:023:52:511
92912995145,4cyclictest189751-21kworker/u8:2+flush-8:021:32:502
92909995142,6cyclictest367920-21latency_hist23:45:001
92907995142,7cyclictest275-21systemd-journal20:40:010
92909995045,4cyclictest128983-21kworker/u8:3+events_unbound19:47:431
3965242500,2sleep20-21swapper/200:12:492
92909994943,4cyclictest189751-21kworker/u8:2+events_unbound21:42:501
92909994943,4cyclictest179068-21kworker/u8:3+events_unbound21:00:011
92907994940,6cyclictest253764-21latency_hist21:50:010
3608712480,2sleep3575-21kworker/3:2+events23:37:433
92912994743,3cyclictest372681-21kworker/u8:0+events_unbound00:10:012
92912994742,4cyclictest92114-21kworker/u8:1+events_unbound19:30:002
92909994744,2cyclictest184416-21kworker/u8:1+flush-8:022:37:551
92909994742,3cyclictest184416-21kworker/u8:1+events_unbound20:52:551
92909994740,5cyclictest184416-21kworker/u8:1+flush-8:022:42:561
92909994740,5cyclictest119450-21kworker/u8:0+events_unbound19:37:381
92909994642,3cyclictest328203-21kworker/u8:2+flush-8:000:12:571
92909994641,4cyclictest119450-21kworker/u8:0+events_unbound20:12:531
92909994641,4cyclictest119450-21kworker/u8:0+events_unbound20:12:531
92909994640,4cyclictest248798-21kworker/u8:0+events_unbound22:07:391
92909994540,4cyclictest407830-21kworker/u8:3+flush-8:000:40:001
92912994338,4cyclictest362218-21kworker/u8:3+flush-8:023:37:522
92912994338,4cyclictest208805-21kworker/u8:0+flush-8:021:07:572
92909994338,4cyclictest348313-21kworker/u8:1+events_unbound23:32:441
92909994333,8cyclictest292841-21ntpq22:27:481
92907994337,4cyclictest328203-21kworker/u8:2+events_unbound00:20:010
92912994237,4cyclictest144326-21kworker/u8:3+flush-8:020:02:542
92909994237,3cyclictest134834-21kworker/u8:1+events_unbound19:52:451
92909994236,4cyclictest119450-21kworker/u8:0+events_unbound20:17:521
92909994235,5cyclictest184416-21kworker/u8:1+flush-8:023:12:491
92909994135,5cyclictest78462-21kworker/u8:3+events_unbound19:15:051
92909994135,4cyclictest362218-21kworker/u8:3+events_unbound23:47:521
92909994135,4cyclictest362218-21kworker/u8:3+events_unbound23:47:521
92909994134,5cyclictest92114-21kworker/u8:1+flush-8:019:22:551
92909994134,5cyclictest298306-21kworker/u8:3+events_unbound23:02:481
92909994134,5cyclictest169295-21kworker/u8:0+flush-8:020:32:421
92912994035,4cyclictest278376-21kworker/u8:0+flush-8:022:23:152
92909994034,4cyclictest78462-21kworker/u8:3+events_unbound19:13:071
92909994034,4cyclictest278376-21kworker/u8:0+flush-8:022:57:571
92909994034,4cyclictest179068-21kworker/u8:3+flush-8:020:37:501
92909994032,6cyclictest308868-21kworker/u8:2+flush-8:022:47:481
92909994031,6cyclictest348087-21latency_hist23:25:011
92907994031,7cyclictest278504-21latency_hist22:15:010
92907994031,7cyclictest275-21systemd-journal20:00:000
9290799401,37cyclictest372871-21unin-run23:50:000
9290799401,37cyclictest372871-21unin-run23:50:000
3105592400,5sleep3451irq_work/322:47:433
92909993934,4cyclictest278376-21kworker/u8:0+events_unbound22:17:531
92909993934,3cyclictest92114-21kworker/u8:1+flush-8:019:32:471
92909993933,4cyclictest372681-21kworker/u8:0+events_unbound00:02:411
92909993932,5cyclictest328203-21kworker/u8:2+flush-8:000:17:561
92907993929,8cyclictest275-21systemd-journal20:30:000
92907993929,8cyclictest224788-21diskstats21:22:400
2610992390,7sleep3451irq_work/321:57:473
92909993833,4cyclictest248798-21kworker/u8:0+events_unbound21:47:451
92909993833,4cyclictest219496-21kworker/u8:0+flush-8:021:22:481
92909993832,4cyclictest288523-21kworker/u8:2+flush-8:022:32:431
92909993831,5cyclictest372681-21kworker/u8:0+events_unbound00:32:491
92909993831,5cyclictest224083-21latency_hist21:20:011
92909993831,5cyclictest184416-21kworker/u8:1+flush-8:022:00:001
92912993733,3cyclictest342942-21kworker/u8:3+flush-8:023:22:522
92912993726,8cyclictest275-21systemd-journal19:57:482
92909993732,4cyclictest348313-21kworker/u8:1+flush-8:023:37:411
92909993731,4cyclictest110856-21irqrtprio19:27:441
92907993731,4cyclictest184416-21kworker/u8:1+flush-8:022:17:420
92907993725,9cyclictest407955-21latency_hist00:25:010
92912993631,4cyclictest271101-21kworker/u8:2+flush-8:022:07:502
92909993630,5cyclictest189751-21kworker/u8:2+flush-8:021:12:511
92907993627,7cyclictest254180-21/usr/sbin/munin21:52:490
92912993530,3cyclictest119450-21kworker/u8:0+flush-8:019:52:532
92909993530,4cyclictest348313-21kworker/u8:1+flush-8:023:27:401
92909993530,4cyclictest184416-21kworker/u8:1+flush-8:020:47:441
92909993530,3cyclictest184416-21kworker/u8:1+flush-8:021:02:451
92909993528,5cyclictest213916-21latency_hist21:10:011
92907993530,4cyclictest179068-21kworker/u8:3+flush-8:020:52:430
92907993526,7cyclictest194114-21latency_hist20:50:010
92909993428,4cyclictest228867-21kworker/u8:3+flush-8:021:37:461
92907993430,3cyclictest169295-21kworker/u8:0+flush-8:020:34:340
92907993424,8cyclictest118837-21ntpq19:32:510
92912993328,4cyclictest412827-21kworker/u8:1+events_unbound00:37:572
92912993328,4cyclictest348313-21kworker/u8:1+events_unbound23:32:462
92912993328,4cyclictest119450-21kworker/u8:0+flush-8:019:42:562
92909993328,4cyclictest333232-21kworker/u8:0+events_unbound23:17:471
92909993328,4cyclictest184416-21kworker/u8:1+events_unbound23:08:211
92907993325,6cyclictest338570-21/usr/sbin/munin23:17:540
92912993228,3cyclictest144326-21kworker/u8:3+events_unbound20:07:572
92912993228,3cyclictest119450-21kworker/u8:0+events_unbound20:17:542
92912993228,2cyclictest184416-21kworker/u8:1+events_unbound20:47:562
883352322,3sleep20-21swapper/219:07:562
92907993122,7cyclictest275-21systemd-journal19:15:000
92907993121,8cyclictest120253-21diskstats19:37:390
92912993026,2cyclictest348616-21aten2_r5power_p23:27:372
92912993025,4cyclictest169295-21kworker/u8:0+flush-8:020:33:312
92909993026,3cyclictest238942-21systemd-run21:35:001
92909993025,4cyclictest278376-21kworker/u8:0+flush-8:022:22:541
92907993023,5cyclictest323336-21latency_hist23:00:010
92907993021,7cyclictest293610-21latency_hist22:30:010
92912992924,4cyclictest333232-21kworker/u8:0+flush-8:023:17:502
92912992923,4cyclictest298306-21kworker/u8:3+events_unbound22:50:082
92907992924,4cyclictest129686-21systemd-run19:45:000
92912992823,4cyclictest348313-21kworker/u8:1+events_unbound23:47:562
92912992823,4cyclictest348313-21kworker/u8:1+events_unbound23:47:562
92912992822,4cyclictest124497-21apt-get19:40:002
92909992824,3cyclictest228867-21kworker/u8:3+flush-8:021:27:481
92909992822,4cyclictest144326-21kworker/u8:3+flush-8:020:07:481
92907992818,8cyclictest275-21systemd-journal21:40:000
92912992722,3cyclictest169990-21fschecks_count20:27:412
92912992719,7cyclictest184416-21kworker/u8:1+flush-8:021:27:582
92909992723,3cyclictest328203-21kworker/u8:2+events_unbound00:27:401
92907992724,2cyclictest327458-21needreboot23:02:480
92907992724,2cyclictest169268-21cron20:24:590
92907992722,4cyclictest160020-21cpuspeed_turbos20:17:400
92907992722,3cyclictest78462-21kworker/u8:3+flush-8:019:17:550
92907992721,4cyclictest188941-21unixbench_singl20:42:560
92907992719,6cyclictest275-21systemd-journal23:28:260
92907992715,10cyclictest110386-21diskstats19:27:400
2162742270,3sleep3575-21kworker/3:2+events21:12:453
92912992623,2cyclictest199779-21fschecks_count20:57:402
92912992623,2cyclictest164822-21aten2_r5power_c20:22:392
92912992621,4cyclictest248798-21kworker/u8:0+flush-8:021:48:122
92912992621,4cyclictest184416-21kworker/u8:1+flush-8:021:58:312
92912992621,3cyclictest278314-21unixbench_singl22:12:582
92912992621,3cyclictest130146-21aten2_r5power_c19:47:372
9291299260,9cyclictest3112101kthreadcore22:47:442
92907992622,3cyclictest288646-21cron22:25:000
92909992520,4cyclictest154245-21kworker/u8:1+flush-8:020:22:491
92907992523,1cyclictest0-21swapper/023:52:550
92907992521,3cyclictest347826-21grep23:22:550
92907992521,3cyclictest333949-21fschecks_count23:12:400
92907992521,3cyclictest328203-21kworker/u8:2+flush-8:000:12:490
92907992520,4cyclictest401-21dbus-daemon19:50:000
92907992518,5cyclictest372681-21kworker/u8:0+flush-8:000:05:310
92907992511,12cyclictest259514-21grep21:57:420
883342254,14sleep10-21swapper/119:07:551
37273222517,5sleep30-21swapper/323:48:403
37273222517,5sleep30-21swapper/323:48:403
3153642250,6sleep1281rcuc/122:52:471
92912992421,2cyclictest244636-21fschecks_time21:42:422
92912992421,2cyclictest119344-21unixbench_singl19:32:572
92912992420,3cyclictest0-21swapper/222:45:002
92912992419,3cyclictest318904-21aten2_r5power_p22:57:392
9291299240,11cyclictest224543-21taskset21:22:372
9290799242,21cyclictest0-21swapper/023:07:550
92907992421,2cyclictest92901-21cyclictest22:37:560
92907992420,3cyclictest0-21swapper/020:07:580
92907992419,3cyclictest0-21swapper/023:45:000
92907992419,3cyclictest0-21swapper/000:00:000
92907992415,7cyclictest145209-21diskstats20:02:380
9290799241,2cyclictest571-21snmpd00:32:550
9291299233,15cyclictest333658-21apt-get23:12:382
9291299232,15cyclictest298996-21apt-get22:37:392
92912992320,2cyclictest268384-21grep22:02:582
92912992320,2cyclictest268384-21grep22:02:582
92912992318,3cyclictest402738-21apt-get00:20:032
92912992318,3cyclictest367902-21cron23:45:002
92912992318,3cyclictest348313-21kworker/u8:1+flush-8:023:55:062
92912992318,3cyclictest109474-21unixbench_singl19:22:572
92912992317,5cyclictest179068-21kworker/u8:3+events_unbound20:50:242
92912992317,4cyclictest179068-21kworker/u8:3+flush-8:020:40:342
92912992310,2cyclictest0-21swapper/223:02:042
92907992321,1cyclictest0-21swapper/022:42:560
92907992318,4cyclictest159095-21unixbench_multi20:12:560
92907992318,4cyclictest159095-21unixbench_multi20:12:560
92907992318,3cyclictest0-21swapper/020:57:370
92907992317,4cyclictest106162-21grep19:22:450
92907992313,8cyclictest312721-21ntpq22:47:480
92912992219,2cyclictest294371-21fschecks_time22:32:402
92912992218,3cyclictest184416-21kworker/u8:1+events_unbound21:03:332
92912992217,4cyclictest97851-21idleruntime-cro19:15:002
92907992219,2cyclictest0-21swapper/000:02:560
92907992218,2cyclictest0-21swapper/021:27:550
92907992217,4cyclictest224086-21munin-run21:20:000
92907992217,4cyclictest213855-21kworker/u8:3+flush-8:021:11:140
92907992217,3cyclictest362683-21unixbench_singl23:37:530
9291299212,14cyclictest398231-21apt-get00:17:382
92912992117,3cyclictest104573-21unixbench_singl19:18:062
92907992117,2cyclictest0-21swapper/022:02:420
92907992117,2cyclictest0-21swapper/022:02:420
92907992117,2cyclictest0-21swapper/000:37:580
92907992116,4cyclictest333232-21kworker/u8:0+events_unbound23:32:550
92907992110,3cyclictest0-21swapper/021:01:170
9290799202,1cyclictest0-21swapper/021:07:560
92907992017,2cyclictest139578-21munin-plugin-st19:55:000
92907992016,2cyclictest0-21swapper/021:32:510
92907992015,4cyclictest0-21swapper/022:52:460
92907992014,4cyclictest1-21systemd22:08:110
3024182200,2sleep3481ktimers/322:37:483
9291299191,8cyclictest2801292kthreadcore22:17:452
92912991915,3cyclictest254317-21aten2_r5power_v21:52:362
92912991915,2cyclictest218846-21unixbench_multi21:12:562
92912991915,2cyclictest0-21swapper/220:40:002
9291299182,12cyclictest158624-21ntpq20:12:502
9291299182,12cyclictest158624-21ntpq20:12:502
92912991814,2cyclictest0-21swapper/200:02:452
92912991812,4cyclictest240475-21kthreadcore21:37:462
92912991810,6cyclictest291920-21tr22:27:462
926522183,9sleep30-21swapper/319:08:503
9291299172,5cyclictest0-21swapper/200:32:452
9291299172,11cyclictest408646-21diskstats00:27:402
9291299168,6cyclictest375754-21tr23:52:462
92912991610,4cyclictest0-21swapper/223:07:482
2894402144,6sleep30-21swapper/322:27:423
3968892132,7sleep30-21swapper/300:12:513
3769962132,7sleep30-21swapper/323:52:493
3287112132,7sleep30-21swapper/323:07:383
3225972132,7sleep30-21swapper/322:57:513
2488162132,7sleep30-21swapper/321:45:003
1886892132,8sleep30-21swapper/320:42:563
1751972132,7sleep30-21swapper/320:32:453
1686142132,7sleep30-21swapper/320:22:533
4181032122,6sleep30-21swapper/300:35:143
4170962123,5sleep30-21swapper/300:32:503
4083722122,6sleep30-21swapper/300:27:363
3991132121,7sleep30-21swapper/300:17:453
3887172122,6sleep30-21swapper/300:07:463
3629462122,5sleep30-21swapper/323:40:013
3539122122,6sleep30-21swapper/323:32:423
3389682122,6sleep30-21swapper/323:17:423
3342892122,6sleep30-21swapper/323:12:443
3243202122,6sleep30-21swapper/323:02:453
2948292122,6sleep30-21swapper/322:32:433
2877562122,6sleep30-21swapper/322:22:483
2691202122,6sleep30-21swapper/322:07:393
2433772122,6sleep30-21swapper/321:37:583
2371462122,6sleep30-21swapper/321:32:483
2083162122,6sleep30-21swapper/321:02:543
1904752122,6sleep30-21swapper/320:47:463
1823132122,6sleep30-21swapper/320:37:463
1597962122,6sleep30-21swapper/320:17:403
1552342122,6sleep30-21swapper/320:12:423
1552342122,6sleep30-21swapper/320:12:423
1449342124,5sleep30-21swapper/320:02:353
1438622122,6sleep30-21swapper/319:57:533
1352862122,6sleep30-21swapper/319:52:403
1202462122,6sleep30-21swapper/319:37:393
1157942122,6sleep30-21swapper/319:32:453
1106272122,6sleep30-21swapper/319:27:443
1089342122,6sleep30-21swapper/319:22:503
1001612122,6sleep30-21swapper/319:17:003
930452112,5sleep30-21swapper/319:10:023
4033022112,6sleep30-21swapper/300:21:103
3832152112,6sleep30-21swapper/300:01:443
3780872112,6sleep30-21swapper/323:55:303
3483772112,5sleep30-21swapper/323:25:563
3435462112,5sleep30-21swapper/323:22:353
3138592112,5sleep30-21swapper/322:52:383
3035482112,5sleep30-21swapper/322:40:023
2738762113,5sleep30-21swapper/322:11:243
2636382112,5sleep30-21swapper/322:00:013
2636382112,5sleep30-21swapper/322:00:013
2541792112,5sleep30-21swapper/321:52:343
2495192112,5sleep30-21swapper/321:47:413
2290512112,5sleep30-21swapper/321:25:013
2243392112,6sleep30-21swapper/321:20:203
2193342112,5sleep30-21swapper/321:15:023
2091802112,5sleep30-21swapper/321:05:033
1992932112,6sleep30-21swapper/320:55:043
1943342112,5sleep30-21swapper/320:50:023
1695232112,6sleep30-21swapper/320:25:183
1499262112,5sleep30-21swapper/320:07:413
1254352112,6sleep30-21swapper/319:42:413
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional