You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-17 - 11:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Thu Sep 17, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19250995549,4cyclictest138738-21apt-get21:05:130
19250995038,11cyclictest24620-21apt-get19:15:130
19252994744,2cyclictest148729-21kworker/u8:1+flush-259:022:10:241
19252994642,3cyclictest154251-21apt-get21:20:101
19252994037,2cyclictest65802-21kworker/u8:2+flush-259:020:10:241
19250993932,5cyclictest294362-21https23:35:000
19256993835,2cyclictest174612-21kworker/u8:2+flush-259:022:55:252
19256993835,2cyclictest174612-21kworker/u8:2+flush-259:022:55:252
19260993733,3cyclictest232118-21kworker/u8:1+events_unbound23:05:253
19256993732,3cyclictest14068-21kworker/u8:1+flush-259:019:50:182
19252993734,2cyclictest200466-21kworker/u8:3+flush-259:022:05:241
19250993731,4cyclictest310702-21diskstats23:50:140
19256993430,3cyclictest14068-21kworker/u8:1+events_unbound19:10:172
19260993328,3cyclictest346581-21latency_hist00:25:003
19252993330,2cyclictest284034-21kworker/u8:4+events_unbound00:25:011
19250993326,5cyclictest91657-21gpgv20:20:000
19260993229,2cyclictest174612-21kworker/u8:2+flush-259:022:20:203
19256993229,2cyclictest226753-21kworker/u8:3+flush-259:022:34:592
19256993228,3cyclictest284034-21kworker/u8:4+flush-259:023:45:012
19252993228,2cyclictest195524-21apt-get22:00:101
19250993125,4cyclictest149050-21apt-get21:15:120
19250993124,5cyclictest336606-21diskstats00:15:130
19260993027,2cyclictest112679-21kworker/u8:3+flush-259:021:25:243
19260993026,3cyclictest174575-21latency_hist21:40:013
19260993024,4cyclictest70042-21ntpq19:55:203
19256993027,2cyclictest273871-21kworker/u8:2+flush-259:023:35:162
19256993024,4cyclictest97631-21diskstats20:25:142
19252993025,3cyclictest352163-21latency_hist00:30:011
19250993025,4cyclictest232118-21kworker/u8:1+flush-259:022:45:170
19256992926,2cyclictest284033-21kworker/u8:1+flush-259:023:25:242
19252992925,3cyclictest4174833-21kworker/u8:0+flush-259:019:25:131
19250992925,3cyclictest335480-21sshd21:54:330
19250992924,4cyclictest305409-21apt-get23:45:110
19256992825,2cyclictest174612-21kworker/u8:2+flush-259:022:45:252
19252992824,3cyclictest4174833-21kworker/u8:0+events_unbound19:44:361
19252992821,6cyclictest169444-21latency_hist21:35:001
19250992823,4cyclictest273871-21kworker/u8:2+flush-259:000:00:160
19250992817,9cyclictest335557-21systemd-journal23:02:330
19260992724,2cyclictest65802-21kworker/u8:2+flush-259:020:40:003
19252992724,2cyclictest112679-21kworker/u8:3+flush-259:021:15:251
19252992724,2cyclictest112679-21kworker/u8:3+events_unbound21:28:351
19252992723,3cyclictest4174833-21kworker/u8:0+flush-259:020:40:181
19252992723,3cyclictest279081-21/usr/sbin/munin23:20:171
19252992719,6cyclictest335557-21systemd-journal23:30:001
19250992712,13cyclictest226708-21turbostat22:29:590
19260992623,2cyclictest174612-21kworker/u8:2+flush-259:021:55:003
19256992622,3cyclictest112679-21kworker/u8:3+flush-259:021:45:222
19256992622,3cyclictest112679-21kworker/u8:3+events_unbound21:15:132
19256992619,5cyclictest147875-21ntpq21:10:212
19252992623,2cyclictest336222-21kworker/u8:2+events_unbound00:18:371
19250992620,4cyclictest174612-21kworker/u8:2+flush-259:022:55:180
19250992620,4cyclictest174612-21kworker/u8:2+flush-259:022:55:170
19250992619,5cyclictest315901-21diskstats23:55:130
19260992521,2cyclictest50489-21/usr/sbin/munin19:40:223
19256992522,2cyclictest65802-21kworker/u8:2+flush-259:020:05:232
19256992521,3cyclictest4174833-21kworker/u8:0+flush-259:020:00:162
19252992522,2cyclictest4174833-21kworker/u8:0+events_unbound20:19:541
19252992522,2cyclictest310507-21kworker/u8:0+events_unbound00:03:281
19252992522,2cyclictest174612-21kworker/u8:2+events_unbound23:00:011
19252992522,2cyclictest174612-21kworker/u8:2+events_unbound23:00:001
19252992522,2cyclictest112679-21kworker/u8:3+flush-259:021:40:201
19252992521,3cyclictest284034-21kworker/u8:4+events_unbound23:55:371
19252992520,4cyclictest335557-21systemd-journal21:55:331
19250992521,2cyclictest128429-21apt-get20:55:120
19250992519,4cyclictest362714-21gpgv00:40:010
19260992421,2cyclictest174612-21kworker/u8:2+flush-259:022:55:273
19260992421,2cyclictest174612-21kworker/u8:2+flush-259:022:55:263
19260992420,3cyclictest112679-21kworker/u8:3+events_unbound21:45:173
19260992420,2cyclictest509-21rs:main3
19260992419,4cyclictest112918-21apt-get20:40:113
19256992420,3cyclictest284034-21kworker/u8:4+flush-259:000:15:012
19256992420,3cyclictest148729-21kworker/u8:1+flush-259:022:20:222
19252992420,3cyclictest4174833-21kworker/u8:0+events_unbound19:20:121
19252992420,3cyclictest232118-21kworker/u8:1+events_unbound23:00:231
19252992419,4cyclictest71222-21apt-get20:00:111
19250992418,4cyclictest143893-21apt-get21:10:130
19260992320,2cyclictest284034-21kworker/u8:4+flush-259:023:45:173
19260992319,3cyclictest149024-21/usr/sbin/munin21:15:203
19260992318,4cyclictest362512-21apt-get00:40:003
19256992320,2cyclictest310507-21kworker/u8:0+flush-259:000:20:172
19252992320,2cyclictest509-21rs:main1
19252992320,2cyclictest310507-21kworker/u8:0+events_unbound00:10:181
19252992320,2cyclictest174612-21kworker/u8:2+events_unbound22:47:341
19252992320,2cyclictest112679-21kworker/u8:3+flush-259:021:10:251
19252992319,3cyclictest148729-21kworker/u8:1+events_unbound22:24:341
19252992318,4cyclictest226974-21apt-get22:30:111
19252992317,4cyclictest14068-21kworker/u8:1+flush-259:019:15:201
19252992316,5cyclictest81314-21latency_hist20:10:001
19250992318,4cyclictest4174833-21kworker/u8:0+flush-259:020:40:150
19260992219,2cyclictest4174833-21kworker/u8:0+events_unbound19:40:013
19256992219,2cyclictest148729-21kworker/u8:1+flush-259:021:35:192
19256992218,3cyclictest268477-21idleruntime-cro23:09:592
19256992217,4cyclictest335557-21systemd-journal00:20:002
19252992219,3cyclictest284034-21kworker/u8:4+events_unbound00:05:021
19252992219,2cyclictest284034-21kworker/u8:4+events_unbound23:45:171
19252992218,3cyclictest273871-21kworker/u8:2+events_unbound23:15:241
19252992218,3cyclictest14068-21kworker/u8:1+events_unbound19:50:201
19252992218,3cyclictest112679-21kworker/u8:3+events_unbound21:03:361
19252992217,4cyclictest174612-21kworker/u8:2+events_unbound22:40:201
19250992218,2cyclictest0-21swapper/022:34:590
19250992216,4cyclictest347111-21cat00:25:000
1925099220,19cyclictest1-21systemd19:46:330
19260992118,2cyclictest273871-21kworker/u8:2+flush-259:023:55:163
19260992118,2cyclictest148729-21kworker/u8:1+events_unbound21:55:133
19260992118,2cyclictest112679-21kworker/u8:3+events_unbound21:40:223
19260992117,3cyclictest65802-21kworker/u8:2+events_unbound20:05:133
19256992118,2cyclictest14068-21kworker/u8:1+events_unbound19:35:232
19256992118,2cyclictest112679-21kworker/u8:3+events_unbound20:45:182
19256992117,3cyclictest65802-21kworker/u8:2+flush-259:020:55:252
19256992117,3cyclictest4174833-21kworker/u8:0+flush-259:019:40:212
19252992117,3cyclictest273871-21kworker/u8:2+flush-259:023:35:201
19252992117,3cyclictest247349-21kworker/u8:3+events_unbound23:11:341
19252992117,3cyclictest232118-21kworker/u8:1+flush-259:023:05:151
19252992116,4cyclictest284033-21kworker/u8:1+events_unbound23:30:341
19250992114,5cyclictest137583-21ntpq21:00:200
19250992113,5cyclictest330998-21latency_hist00:10:010
19250992112,8cyclictest61150irq/133-enp0s29f1:tx-019:10:170
1925099211,17cyclictest195288-21unin-run22:00:000
19260992017,2cyclictest284034-21kworker/u8:4+flush-259:000:05:203
19260992016,3cyclictest335557-21systemd-journal20:30:003
19260992016,3cyclictest294369-21gpgv23:35:013
19256992017,2cyclictest112679-21kworker/u8:3+flush-259:021:00:152
19256992016,3cyclictest112679-21kworker/u8:3+events_unbound21:25:232
19256992014,4cyclictest335557-21systemd-journal22:05:132
19252992016,3cyclictest4174833-21kworker/u8:0+events_unbound20:35:151
19252992016,3cyclictest346351-21kworker/u8:1+events_unbound00:30:271
19252992016,3cyclictest14068-21kworker/u8:1+events_unbound19:49:481
19252992016,3cyclictest14068-21kworker/u8:1+events_unbound19:35:151
19252992016,3cyclictest112679-21kworker/u8:3+flush-259:020:45:251
19252992015,4cyclictest112679-21kworker/u8:3+flush-259:020:50:391
19252992015,4cyclictest112679-21kworker/u8:3+flush-259:020:50:381
19252992014,4cyclictest0-21swapper/120:25:001
1925099208,10cyclictest232267-21tr22:35:140
1925099206,2cyclictest0-21swapper/020:10:000
1925099206,13cyclictest97443-21/usr/sbin/munin20:25:190
1925099204,2cyclictest346580-21turbostat.cron00:25:010
1925099202,2cyclictest216196-21apt-get22:20:130
1925099201,6cyclictest335557-21systemd-journal23:45:010
1925099201,5cyclictest336177-21idleruntime-cro00:15:000
1925099201,4cyclictest184935-21idleruntime-cro21:50:000
1925099201,4cyclictest179765-21cron21:45:000
1925099201,16cyclictest335557-21systemd-journal19:20:330
1925099200,5cyclictest50267-21sleep19:40:000
19260991917,2cyclictest273871-21kworker/u8:2+flush-259:023:15:213
19260991917,2cyclictest273664-21kworker/u8:0+flush-259:023:20:183
19260991916,2cyclictest273871-21kworker/u8:2+events_unbound00:00:193
19260991916,2cyclictest174612-21kworker/u8:2+flush-259:022:00:233
19260991915,3cyclictest112679-21kworker/u8:3+flush-259:020:45:203
19260991915,3cyclictest112679-21kworker/u8:3+events_unbound20:50:213
19260991915,3cyclictest112679-21kworker/u8:3+events_unbound20:50:203
19256991916,2cyclictest184976-21kworker/u8:0+flush-259:021:50:152
19256991915,3cyclictest148729-21kworker/u8:1+flush-259:022:10:152
19252991916,2cyclictest174612-21kworker/u8:2+events_unbound22:25:131
19252991915,3cyclictest65802-21kworker/u8:2+flush-259:020:30:341
19252991914,4cyclictest112679-21kworker/u8:3+events_unbound21:45:181
1925099191,5cyclictest335557-21systemd-journal23:19:590
1925099191,4cyclictest118374-21fschecks_time20:45:150
1925099191,4cyclictest107484-21cron20:35:000
19250991911,7cyclictest60350irq/132-enp0s29f1:rx-022:50:210
1925099190,5cyclictest75377-21ps20:00:230
1925099190,5cyclictest273631-21awk23:15:000
1925099190,4cyclictest70964-21latency_hist20:00:000
19260991815,2cyclictest4174833-21kworker/u8:0+events_unbound19:45:193
19260991814,3cyclictest4174833-21kworker/u8:0+flush-259:019:20:203
19260991814,3cyclictest310540-21apt-get23:50:103
19260991814,3cyclictest210780-21latency_hist22:15:023
19260991814,3cyclictest174612-21kworker/u8:2+flush-259:022:05:183
19256991814,3cyclictest4174833-21kworker/u8:0+flush-259:020:40:232
19256991814,3cyclictest310507-21kworker/u8:0+flush-259:000:25:192
19256991814,3cyclictest273664-21kworker/u8:0+events_unbound23:20:132
19256991814,2cyclictest210821-21kworker/u8:0+events_unbound22:35:182
19256991813,3cyclictest362889-21cat00:40:012
19252991814,3cyclictest232118-21kworker/u8:1+flush-259:022:50:281
19252991814,3cyclictest148729-21kworker/u8:1+events_unbound21:39:341
1925099181,4cyclictest357360-21systemd-run00:35:010
1925099181,4cyclictest335557-21systemd-journal19:35:000
19250991814,2cyclictest148729-21kworker/u8:1+events_unbound21:35:180
1925099181,3cyclictest335557-21systemd-journal20:40:010
19250991812,4cyclictest86437-21sh20:15:000
1925099180,4cyclictest97205-21latency_hist20:25:010
1925099180,4cyclictest35398-21sed19:25:140
1925099180,4cyclictest206287-21irqcore22:10:160
1925099180,4cyclictest169435-21awk21:35:000
1925099180,4cyclictest155270-21grep21:20:160
1925099180,3cyclictest335557-21systemd-journal22:44:590
19260991714,2cyclictest65802-21kworker/u8:2+events_unbound20:15:163
19260991714,2cyclictest273871-21kworker/u8:2+flush-259:023:35:213
19260991714,2cyclictest14068-21kworker/u8:1+events_unbound19:30:243
19260991713,3cyclictest4174833-21kworker/u8:0+events_unbound19:25:143
19260991713,3cyclictest232118-21kworker/u8:1+flush-259:022:50:203
19260991713,3cyclictest143616-21latency_hist21:10:003
19256991714,2cyclictest346351-21kworker/u8:1+events_unbound00:30:152
19256991713,3cyclictest65802-21kworker/u8:2+events_unbound20:20:182
19256991713,3cyclictest112679-21kworker/u8:3+events_unbound21:40:242
19256991712,4cyclictest65802-21kworker/u8:2+events_unbound20:15:222
19252991714,2cyclictest232118-21kworker/u8:1+events_unbound22:35:231
19252991713,3cyclictest4174833-21kworker/u8:0+events_unbound19:59:351
19252991712,4cyclictest284034-21kworker/u8:4+events_unbound23:50:011
1925099173,3cyclictest63544-21cut19:50:190
1925099173,3cyclictest280644-21cut23:20:150
1925099171,4cyclictest285499-21kthreadcore23:25:180
1925099171,4cyclictest215070-21/usr/sbin/munin22:15:250
1925099170,4cyclictest267575-21chrt23:05:200
1925099170,4cyclictest159694-21/usr/sbin/munin21:25:150
19260991613,3cyclictest112679-21kworker/u8:3+events_unbound21:20:143
19260991613,2cyclictest284033-21kworker/u8:1+flush-259:023:25:163
19260991613,2cyclictest232118-21kworker/u8:1+events_unbound22:49:333
19256991613,2cyclictest242190-21cron22:45:002
19256991613,2cyclictest174612-21kworker/u8:2+events_unbound22:25:242
19256991612,3cyclictest65802-21kworker/u8:2+events_unbound20:10:132
19256991612,3cyclictest4174833-21kworker/u8:0+events_unbound19:20:162
19256991612,3cyclictest247349-21kworker/u8:3+events_unbound23:10:212
19256991612,3cyclictest232118-21kworker/u8:1+flush-259:022:50:152
19256991612,3cyclictest112679-21kworker/u8:3+events_unbound20:50:242
19256991612,3cyclictest112679-21kworker/u8:3+events_unbound20:50:242
19252991613,2cyclictest284034-21kworker/u8:4+flush-259:023:40:211
19252991612,3cyclictest112679-21kworker/u8:3+events_unbound21:05:271
19252991611,4cyclictest148729-21kworker/u8:1+events_unbound22:17:341
1925099162,3cyclictest195524-21apt-get22:00:100
1925099161,4cyclictest299067-21ntpq23:35:230
1925099161,3cyclictest19177-21latency_hist19:10:010
19260991512,2cyclictest14068-21kworker/u8:1+flush-259:019:15:143
19260991512,2cyclictest112679-21kworker/u8:3+flush-259:021:00:183
19260991511,3cyclictest65802-21kworker/u8:2+events_unbound20:03:343
19260991511,3cyclictest174612-21kworker/u8:2+events_unbound22:25:163
19260991511,3cyclictest112679-21kworker/u8:3+events_unbound21:30:183
19256991512,2cyclictest65802-21kworker/u8:2+events_unbound21:05:192
19256991512,2cyclictest39936-21sh19:30:002
19256991512,2cyclictest14068-21kworker/u8:1+events_unbound19:30:142
19256991511,3cyclictest174612-21kworker/u8:2+flush-259:022:00:152
19256991511,3cyclictest148729-21kworker/u8:1+flush-259:021:55:202
19252991513,1cyclictest0-21swapper/100:40:011
19250991513,2cyclictest61150irq/133-enp0s29f1:tx-019:40:160
1926099149,4cyclictest263307-21latency_hist23:05:013
19260991410,3cyclictest210821-21kworker/u8:0+flush-259:022:30:233
19256991411,2cyclictest4174833-21kworker/u8:0+events_unbound20:35:172
19252991411,2cyclictest14068-21kworker/u8:1+flush-259:019:30:251
19252991411,2cyclictest14068-21kworker/u8:1+events_unbound19:10:231
1926099138,4cyclictest65802-21kworker/u8:2+flush-259:021:13:093
19260991310,2cyclictest65802-21kworker/u8:2+flush-259:020:10:163
19260991310,2cyclictest210821-21kworker/u8:0+flush-259:022:35:153
1925699139,3cyclictest148729-21kworker/u8:1+flush-259:021:34:102
1925699139,3cyclictest0-21swapper/221:25:002
19256991310,2cyclictest310621-21cpuspeed_turbos23:50:122
1926099128,3cyclictest174612-21kworker/u8:2+flush-259:022:15:443
1926099128,3cyclictest112679-21kworker/u8:3+events_unbound20:55:273
1926099127,3cyclictest335557-21systemd-journal23:40:303
19260991210,1cyclictest0-21swapper/300:15:013
1925299129,2cyclictest128566-21cpuspeed_turbos20:55:141
1925699118,2cyclictest284034-21kworker/u8:4+events_unbound00:05:252
1925699118,2cyclictest148729-21kworker/u8:1+events_unbound22:15:252
1925699118,2cyclictest14068-21kworker/u8:1+flush-259:019:15:262
1925699118,2cyclictest0-21swapper/220:00:002
1925699117,3cyclictest273871-21kworker/u8:2+events_unbound23:30:192
1925699110,6cyclictest320677-21cmp00:00:012
1925299116,4cyclictest174612-21kworker/u8:2+events_unbound21:53:201
1926099107,1cyclictest0-21swapper/319:50:213
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional