You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-12 - 11:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Sun Jul 12, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
238690994239,2cyclictest477494-21kworker/u8:3+events_unbound23:12:031
238690993330,2cyclictest265059-21kworker/u8:2+events_unbound20:11:351
238690993329,3cyclictest409552-21kworker/u8:1+events_unbound22:53:341
238690993327,4cyclictest335557-21systemd-journal22:35:011
238690993228,3cyclictest514346-21kworker/u8:0+events_unbound23:46:351
238690993128,2cyclictest514346-21kworker/u8:0+events_unbound23:42:451
238690993128,2cyclictest514346-21kworker/u8:0+events_unbound23:42:451
238690993127,3cyclictest409552-21kworker/u8:1+flush-259:023:27:441
238688993125,4cyclictest455838-21latency_hist22:40:010
238690993026,3cyclictest455824-21kworker/u8:0+flush-259:022:40:351
238690993026,3cyclictest429991-21kworker/u8:3+events_unbound22:19:361
238690992925,3cyclictest265059-21kworker/u8:2+events_unbound20:08:351
238690992925,3cyclictest222995-21kworker/u8:3+events_unbound19:35:201
238690992824,3cyclictest409552-21kworker/u8:1+events_unbound22:47:011
238690992824,3cyclictest222995-21kworker/u8:3+events_unbound19:47:361
238690992724,2cyclictest561211-21kworker/u8:2+events_unbound00:25:221
238690992724,2cyclictest477494-21kworker/u8:3+events_unbound00:00:021
238690992724,2cyclictest389086-21kworker/u8:4+events_unbound21:35:231
238693992623,2cyclictest3713660-21kworker/u8:1+flush-259:020:20:162
238693992620,3cyclictest269989-21latency_hist19:40:012
238690992623,2cyclictest477494-21kworker/u8:3+events_unbound23:55:211
238690992623,2cyclictest409552-21kworker/u8:1+events_unbound22:57:421
238690992623,2cyclictest373455-21kworker/u8:2+events_unbound22:00:171
238690992623,2cyclictest3713660-21kworker/u8:1+events_unbound20:24:351
238690992623,2cyclictest3713660-21kworker/u8:1+events_unbound19:25:251
238690992623,2cyclictest311096-21kworker/u8:0+events_unbound21:42:261
238690992623,2cyclictest265059-21kworker/u8:2+events_unbound19:59:361
238690992622,3cyclictest477494-21kworker/u8:3+events_unbound23:51:371
238690992622,3cyclictest429991-21kworker/u8:3+events_unbound22:23:341
238690992622,3cyclictest409552-21kworker/u8:1+flush-259:022:35:351
238690992622,3cyclictest3713660-21kworker/u8:1+events_unbound19:31:301
238688992620,4cyclictest254534-21latency_hist19:25:000
23868899260,14cyclictest460097-21/usr/sbin/munin22:40:200
238693992522,2cyclictest362421-21kworker/u8:1+events_unbound21:10:192
238693992521,3cyclictest311096-21kworker/u8:0+flush-259:022:10:232
238690992522,2cyclictest477494-21kworker/u8:3+events_unbound00:21:381
238690992521,3cyclictest311096-21kworker/u8:0+events_unbound22:10:361
238688992520,4cyclictest498468-21cat23:20:000
238688992519,4cyclictest335557-21systemd-journal22:30:000
238698992419,3cyclictest534904-21latency_hist23:55:003
238698992417,4cyclictest540533-21latency_hist00:00:013
238693992418,4cyclictest534921-21kworker/u8:1+flush-259:000:10:012
238690992421,2cyclictest477494-21kworker/u8:3+events_unbound23:35:141
238690992421,2cyclictest409552-21kworker/u8:1+events_unbound23:02:341
238690992421,2cyclictest373455-21kworker/u8:2+events_unbound21:23:361
238690992421,2cyclictest3713660-21kworker/u8:1+events_unbound20:26:351
238690992421,2cyclictest311096-21kworker/u8:0+events_unbound20:30:281
238690992421,2cyclictest280500-21kworker/u8:0+events_unbound19:50:141
238688992411,12cyclictest335557-21systemd-journal00:24:370
23868899240,23cyclictest1-21systemd21:26:340
23868899240,23cyclictest1-21systemd21:26:340
23868899240,14cyclictest546118-21latency_hist00:05:010
23868899240,14cyclictest449631-21missed_timers22:30:200
23868899240,14cyclictest430541-21entropy22:15:130
23868899240,13cyclictest476544-21/usr/sbin/munin22:55:250
23868899240,13cyclictest387972-21ntpq21:30:210
238698992315,6cyclictest399208-21latency_hist21:45:013
238693992316,5cyclictest465370-21ntpq22:45:242
238690992321,2cyclictest477494-21kworker/u8:3+events_unbound23:30:121
238690992320,2cyclictest477494-21kworker/u8:3+events_unbound00:12:381
238690992320,2cyclictest409552-21kworker/u8:1+events_unbound23:15:301
238690992320,2cyclictest373455-21kworker/u8:2+events_unbound21:57:361
238690992320,2cyclictest222995-21kworker/u8:3+events_unbound19:42:121
238688992317,4cyclictest535299-21cat23:55:000
23868899230,15cyclictest325644-21grep20:30:190
23868899230,14cyclictest505235-21cut23:25:170
23868899230,14cyclictest440448-21latency_hist22:25:010
23868899230,12cyclictest338675-21cut20:45:170
238698992219,2cyclictest3713660-21kworker/u8:1+flush-259:019:25:203
238698992216,4cyclictest466197-21latency_hist22:50:013
238693992216,4cyclictest335557-21systemd-journal20:00:002
238690992219,2cyclictest534921-21kworker/u8:1+events_unbound00:05:191
238690992219,2cyclictest238872-21kworker/u8:4+events_unbound19:10:191
238688992216,5cyclictest471719-21latency_hist22:55:000
23868899220,13cyclictest317382-21cut20:25:180
23868899220,12cyclictest554927-21kthreadcore00:10:180
23868899220,12cyclictest295954-21https20:05:130
238698992117,3cyclictest440638-21/usr/sbin/munin22:25:153
238693992117,3cyclictest265059-21kworker/u8:2+flush-259:020:10:222
238693992114,5cyclictest335557-21systemd-journal21:00:002
238693992114,4cyclictest335557-21systemd-journal00:15:002
238690992118,2cyclictest561211-21kworker/u8:2+events_unbound00:35:281
238690992118,2cyclictest337161-21kworker/u8:4+events_unbound21:06:351
238690992118,2cyclictest337161-21kworker/u8:4+events_unbound20:45:231
23868899218,11cyclictest304975-21needreboot20:10:170
238688992118,2cyclictest316274-21cron20:25:010
238688992116,3cyclictest335557-21systemd-journal21:17:230
23868899210,13cyclictest520037-21/usr/sbin/munin23:40:150
23868899210,13cyclictest520037-21/usr/sbin/munin23:40:140
23868899210,12cyclictest486670-21ld23:05:230
23868899210,12cyclictest365605-21cut21:10:190
23868899210,12cyclictest347004-21sshd20:51:340
23868899210,11cyclictest572181-21chrt00:27:390
23868899210,11cyclictest269168-21ntpq19:35:220
23868899210,11cyclictest248489-21ld19:15:220
23868899210,10cyclictest361851-21ntpq21:05:250
238698992014,4cyclictest556661-21/usr/sbin/munin00:15:143
238698992014,4cyclictest362911-21gpgv21:10:003
238690992017,2cyclictest561211-21kworker/u8:2+events_unbound00:32:521
238690992017,2cyclictest238872-21kworker/u8:4+events_unbound19:15:151
238690992016,3cyclictest477494-21kworker/u8:3+events_unbound23:22:441
23868899209,10cyclictest0-21swapper/020:05:010
23868899204,14cyclictest403934-21awk21:45:260
238688992011,7cyclictest60350irq/132-enp0s29f1:rx-000:15:120
238688992010,8cyclictest60350irq/132-enp0s29f1:rx-000:30:130
23868899200,17cyclictest517022-21grep23:35:180
23868899200,10cyclictest0-21swapper/021:55:000
238698991917,1cyclictest492457-21fwupd23:12:043
238698991916,2cyclictest367994-21kworker/u8:3+flush-259:021:20:003
238698991914,4cyclictest399470-21apt-get21:45:113
238693991916,2cyclictest367994-21kworker/u8:3+events_unbound21:30:152
238693991915,3cyclictest540535-21apt-get00:00:002
238693991915,2cyclictest0-21swapper/220:39:592
238693991914,3cyclictest311096-21kworker/u8:0+flush-259:021:50:002
238693991912,5cyclictest335557-21systemd-journal22:54:592
238690991916,2cyclictest327417-21kworker/u8:2+flush-259:020:37:351
238690991916,2cyclictest316297-21kworker/u8:3+flush-259:020:42:351
23868899199,9cyclictest0-21swapper/019:45:250
23868899195,12cyclictest540934-21wc00:00:000
23868899191,16cyclictest274203-21ntpq19:40:200
23868899190,9cyclictest306415-21diskstats20:15:130
23868899190,17cyclictest529779-21runrttasks23:50:000
23868899190,12cyclictest488097-21fw_forwarded_lo23:10:140
23868899190,10cyclictest481544-21ntpq23:00:190
238698991814,3cyclictest3713660-21kworker/u8:1+flush-259:019:50:013
23869399185,7cyclictest311096-21kworker/u8:0+ext4-rsv-conversion20:50:002
238693991815,1cyclictest0-21swapper/221:20:002
238693991814,3cyclictest3713660-21kworker/u8:1+flush-259:019:25:222
238693991814,2cyclictest0-21swapper/200:40:012
238693991813,3cyclictest244468-21/usr/sbin/munin19:15:222
238690991817,1cyclictest311096-21kworker/u8:0+events_unbound21:04:351
238690991816,2cyclictest265059-21kworker/u8:2+events_unbound20:00:141
238690991815,2cyclictest300820-21kworker/u8:3+events_unbound20:19:351
238690991814,3cyclictest404628-21/usr/sbin/munin21:50:141
238690991814,3cyclictest367994-21kworker/u8:3+events_unbound21:26:491
238690991814,3cyclictest367994-21kworker/u8:3+events_unbound21:26:491
23868899187,10cyclictest0-21swapper/023:20:130
23868899185,11cyclictest239049-21cpuspeed_turbos19:10:120
23868899184,12cyclictest391438-21kthreadcore21:35:160
238688991814,2cyclictest0-21swapper/021:04:590
23868899181,16cyclictest509-21rs:main0
23868899181,15cyclictest465475-21/usr/sbin/munin22:45:260
23868899181,15cyclictest418953-21ntpq22:00:230
23868899180,9cyclictest577824-21apt-get00:35:130
23868899180,9cyclictest290438-21taskset19:58:520
23868899180,9cyclictest259919-21apt-get19:30:140
23868899180,16cyclictest430120-21idleruntime-cro22:15:000
23868899180,16cyclictest351630-21open_inodes20:55:250
23868899180,16cyclictest336042-21ntpq20:40:220
23868899180,16cyclictest326981-21cut20:35:130
23868899180,15cyclictest394327-21apt-get21:40:110
23868899180,13cyclictest255086-21fw_conntrack19:25:170
23868899180,10cyclictest378473-21chrt21:23:390
238690991715,2cyclictest429991-21kworker/u8:3+events_unbound22:25:271
238690991715,2cyclictest311096-21kworker/u8:0+events_unbound21:19:221
23868899170,9cyclictest409938-21diskstats21:55:170
23868899170,15cyclictest550342-21ntpq00:05:200
23868899170,15cyclictest509775-21ps23:30:150
238698991613,2cyclictest487556-21turbostat23:10:013
238698991613,2cyclictest0-21swapper/320:10:013
238698991612,3cyclictest429991-21kworker/u8:3+flush-259:022:45:013
238698991612,3cyclictest373455-21kworker/u8:2+flush-259:022:00:003
23869899160,14cyclictest1-21systemd20:30:013
238693991613,2cyclictest327417-21kworker/u8:2+flush-259:020:50:202
238693991613,1cyclictest0-21swapper/221:25:002
238693991612,3cyclictest373455-21kworker/u8:2+flush-259:022:00:192
238693991612,2cyclictest0-21swapper/223:55:002
238690991613,2cyclictest367994-21kworker/u8:3+events_unbound21:30:031
238690991613,2cyclictest311096-21kworker/u8:0+events_unbound20:55:341
238690991612,3cyclictest311096-21kworker/u8:0+events_unbound22:06:141
23868899160,15cyclictest280841-21hddtemp_smartct19:50:150
23868899160,13cyclictest238617-21latency_hist19:10:010
238698991512,2cyclictest504574-21irqcore23:25:163
238698991510,4cyclictest471714-21apt-get22:55:003
23869399159,4cyclictest378832-21/usr/sbin/munin21:25:182
23869399159,4cyclictest378832-21/usr/sbin/munin21:25:182
238693991512,2cyclictest561569-21sh00:20:002
238693991512,2cyclictest449802-21ntp_kernel_pll_22:30:202
238693991512,2cyclictest363010-21latency_hist21:09:592
238690991512,2cyclictest327417-21kworker/u8:2+events_unbound20:50:261
238690991512,2cyclictest311096-21kworker/u8:0+events_unbound21:45:281
238698991412,1cyclictest0-21swapper/323:50:003
238698991411,2cyclictest451376-21irqcore22:35:163
238698991411,2cyclictest0-21swapper/320:20:013
238698991410,3cyclictest326839-21/usr/sbin/munin20:35:203
238698991410,3cyclictest286083-21tr19:55:163
238693991411,2cyclictest518504-21needreboot23:35:202
238693991411,2cyclictest0-21swapper/220:25:262
238693991411,1cyclictest520764-21kthreadcore23:40:182
238693991411,1cyclictest520764-21kthreadcore23:40:172
238693991410,3cyclictest409552-21kworker/u8:1+flush-259:023:17:362
23869399140,13cyclictest1-21systemd19:50:342
23869099145,2cyclictest0-21swapper/100:15:171
23869099144,2cyclictest0-21swapper/121:10:181
238690991411,2cyclictest3713660-21kworker/u8:1+events_unbound19:23:211
23869899137,5cyclictest523775-21ntpq23:40:213
23869899137,5cyclictest523775-21ntpq23:40:213
23869899137,4cyclictest388849-21latency_hist21:35:013
23869899136,5cyclictest490-21dbus-daemon21:00:003
23869899136,5cyclictest243992-21cp19:15:013
238698991311,1cyclictest394065-21turbostat.cron21:40:003
238698991310,2cyclictest546788-21irqcore00:05:163
238698991310,2cyclictest509436-21cpuspeed_turbos23:30:123
23869399139,3cyclictest265059-21kworker/u8:2+flush-259:020:05:282
23869399138,3cyclictest529766-21idleruntime-cro23:50:002
23869399138,3cyclictest335557-21systemd-journal22:16:552
23869399137,4cyclictest572394-21awk00:30:002
23869399136,5cyclictest562178-21cat00:20:022
238693991310,2cyclictest0-21swapper/223:05:002
238690991310,2cyclictest486537-21needreboot23:05:211
23869899129,2cyclictest577061-21unixbench_multi00:30:253
23869899129,2cyclictest516878-21kthreadcore23:35:183
23869899127,4cyclictest238872-21kworker/u8:4+flush-259:019:19:593
23869899127,3cyclictest567190-21latency_hist00:25:003
23869899121,10cyclictest450698-21awk22:35:003
238698991210,1cyclictest0-21swapper/323:05:003
238698991210,1cyclictest0-21swapper/319:50:193
23869399129,2cyclictest482760-21cpuspeed_turbos23:05:142
23869399129,2cyclictest357590-21latency_hist21:05:002
23869399129,2cyclictest265059-21kworker/u8:2+flush-259:020:00:182
23869399129,2cyclictest0-21swapper/221:50:192
238693991210,1cyclictest0-21swapper/223:20:192
238693991210,1cyclictest0-21swapper/223:15:002
238693991210,1cyclictest0-21swapper/222:40:192
23869899118,2cyclictest373871-21fschecks_count21:20:133
23869899118,2cyclictest0-21swapper/322:54:593
23869899118,1cyclictest0-21swapper/322:05:163
23869899115,5cyclictest321886-21diskstats20:30:143
23869899115,4cyclictest335557-21systemd-journal20:25:013
23869399107,2cyclictest0-21swapper/219:20:192
23869399106,3cyclictest3713660-21kworker/u8:1+events_unbound19:40:162
2386989994,3cyclictest555492-21ntp_00:10:203
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional