You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-21 - 06:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Sun Dec 21, 2025 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4044911994642,3cyclictest4091774-21kworker/u8:4+events_unbound20:35:010
4044911994236,4cyclictest4164175-21kworker/u8:0+events_unbound21:20:170
4044914993932,4cyclictest4091759-21latency_hist19:55:011
4044911993932,2cyclictest4184042-21kworker/u8:4+events_unbound21:40:140
4044914993832,4cyclictest64161-21latency_hist22:35:011
4044911993835,2cyclictest4006506-21kworker/u8:2+flush-259:019:50:000
4044911993835,2cyclictest178979-21kworker/u8:0+events_unbound00:25:240
4044911993835,2cyclictest110905-21kworker/u8:4+flush-259:023:25:240
4044911993832,4cyclictest4133343-21apt-get20:35:140
4044914993733,3cyclictest4055294-21gpgv19:20:001
4044911993734,2cyclictest4055832-21kworker/u8:0+events_unbound19:25:220
4044911993734,2cyclictest116875-21kworker/u8:1+events_unbound00:30:230
404491199371,34cyclictest32864-21munin-run22:05:000
4044911993632,3cyclictest152745-21kworker/u8:2+events_unbound00:20:230
4044922993532,2cyclictest4096934-21kworker/u8:5+flush-259:020:15:193
4044914993530,4cyclictest116887-21/usr/sbin/munin23:25:151
4044911993430,3cyclictest4164175-21kworker/u8:0+flush-259:021:15:270
4044922993327,4cyclictest64345-21cp22:35:013
4044917993330,2cyclictest4189795-21kworker/u8:1+flush-259:021:30:252
4044911993330,2cyclictest4065972-21kworker/u8:3+flush-259:020:20:250
4044911993328,3cyclictest127011-21kworker/u8:5+flush-259:023:50:230
4044922993229,2cyclictest4184042-21kworker/u8:4+flush-259:021:45:163
4044917993229,2cyclictest4189795-21kworker/u8:1+flush-259:021:40:182
4044914993225,5cyclictest167507-21ntpq00:10:211
4044911993229,2cyclictest4039952-21kworker/u8:3+events_unbound19:15:190
4044911993228,3cyclictest64086-21kworker/u8:1+events_unbound22:40:230
4044922993124,5cyclictest193439-21ntpq00:35:213
4044917993127,3cyclictest158473-21kworker/u8:4+flush-259:000:10:182
4044922993024,4cyclictest27891-21/usr/sbin/munin22:00:143
4044914993024,4cyclictest43775-21/usr/sbin/munin22:15:231
4044914993023,5cyclictest90411-21latency_hist23:00:011
4044911993026,3cyclictest4164175-21kworker/u8:0+events_unbound21:15:000
4044911993023,5cyclictest38401-21latency_hist22:10:010
4044922992924,3cyclictest252-21systemd-journal21:55:003
4044922992924,3cyclictest252-21systemd-journal21:54:593
4044922992923,4cyclictest4071379-21/usr/sbin/munin19:35:173
4044922992923,4cyclictest4071379-21/usr/sbin/munin19:35:163
4044917992925,3cyclictest4153833-21kworker/u8:1+flush-259:021:05:012
4044911992926,2cyclictest90050-21kworker/u8:1+events_unbound23:00:180
4044911992926,2cyclictest53538-21kworker/u8:4+flush-259:022:55:210
4044911992926,2cyclictest4127932-21kworker/u8:0+events_unbound20:45:240
4044911992924,3cyclictest4189795-21kworker/u8:1+flush-259:021:30:180
4044922992822,4cyclictest172668-21ntpq00:15:203
4044911992825,2cyclictest4184042-21kworker/u8:4+flush-259:021:45:200
4044911992825,2cyclictest116875-21kworker/u8:1+flush-259:023:55:230
4044911992824,3cyclictest110905-21kworker/u8:4+flush-259:023:35:280
4044911992724,2cyclictest4117844-21kworker/u8:1+events_unbound20:40:140
4044911992724,2cyclictest1412-21kworker/u8:0+flush-259:021:50:260
4044911992724,2cyclictest1412-21kworker/u8:0+flush-259:021:50:260
4044922992623,2cyclictest4055832-21kworker/u8:0+flush-259:020:00:153
4044917992620,4cyclictest163214-21latency_hist00:10:002
4044914992620,4cyclictest132466-21apt-get23:40:121
4044914992620,4cyclictest104971-21ntpq23:10:201
4044911992622,3cyclictest4148481-21kworker/u8:3+flush-259:021:00:220
4044911992622,3cyclictest11250-21kworker/u8:3+events_unbound22:25:250
4044911992622,3cyclictest11250-21kworker/u8:3+events_unbound22:10:140
4044922992522,2cyclictest116875-21kworker/u8:1+flush-259:023:35:203
4044922992517,4cyclictest252-21systemd-journal00:10:013
4044922992517,4cyclictest252-21systemd-journal00:10:003
4044917992522,2cyclictest32424-21kworker/u8:2+flush-259:022:05:022
4044917992519,5cyclictest178693-21apt-get00:24:552
4044911992522,2cyclictest4006506-21kworker/u8:2+flush-259:019:55:180
4044911992521,3cyclictest48438-21kworker/u8:2+flush-259:022:20:260
4044911992521,3cyclictest48438-21kworker/u8:2+events_unbound23:30:180
4044911992521,3cyclictest4143128-21kworker/u8:2+events_unbound21:25:010
4044917992421,2cyclictest75133-21kworker/u8:0+flush-259:023:05:262
4044917992421,2cyclictest153102-21kworker/u8:3+flush-259:000:05:002
4044917992420,3cyclictest4091774-21kworker/u8:4+flush-259:020:15:152
4044914992419,3cyclictest4117844-21kworker/u8:1+flush-259:020:45:181
4044914992418,4cyclictest4102575-21diskstats20:05:151
4044911992421,2cyclictest4065972-21kworker/u8:3+flush-259:019:30:220
4044922992320,2cyclictest11250-21kworker/u8:3+flush-259:022:10:243
4044922992317,4cyclictest252-21systemd-journal00:32:533
4044917992319,3cyclictest4164175-21kworker/u8:0+flush-259:021:15:192
4044917992319,3cyclictest4065972-21kworker/u8:3+flush-259:019:35:002
4044917992319,3cyclictest4060791-21latency_hist19:25:012
4044914992318,3cyclictest1-21systemd22:45:001
4044914992316,5cyclictest4065947-21latency_hist19:30:011
4044911992320,2cyclictest4127932-21kworker/u8:0+events_unbound20:55:230
4044911992320,2cyclictest4091774-21kworker/u8:4+flush-259:020:05:210
4044911992320,2cyclictest4045083-21kworker/u8:4+flush-259:019:35:220
4044911992320,2cyclictest4045083-21kworker/u8:4+flush-259:019:35:210
4044911992320,2cyclictest127011-21kworker/u8:5+events_unbound23:40:150
4044922992219,2cyclictest116875-21kworker/u8:1+flush-259:000:00:173
4044922992218,3cyclictest75133-21kworker/u8:0+flush-259:023:15:253
4044922992218,3cyclictest4164175-21kworker/u8:0+flush-259:021:05:253
4044922992218,3cyclictest127011-21kworker/u8:5+events_unbound23:50:143
4044917992219,2cyclictest11250-21kworker/u8:3+events_unbound22:15:012
4044911992220,2cyclictest4045083-21kworker/u8:4+events_unbound19:20:250
4044911992219,2cyclictest4006506-21kworker/u8:2+flush-259:019:40:200
4044911992219,2cyclictest22514-21kworker/u8:1+events_unbound22:15:210
4044911992219,2cyclictest137378-21kworker/u8:0+events_unbound00:03:380
4044911992215,5cyclictest80063-21latency_hist22:50:010
4044922992116,4cyclictest252-21systemd-journal20:25:013
4044922992116,3cyclictest1412-21kworker/u8:0+events_unbound21:35:243
4044917992117,3cyclictest4153833-21kworker/u8:1+flush-259:021:20:242
4044917992117,3cyclictest4006506-21kworker/u8:2+flush-259:019:40:202
4044917992117,3cyclictest11250-21kworker/u8:3+events_unbound21:55:212
4044917992116,3cyclictest4086591-21latency_hist19:50:002
4044914992115,4cyclictest48438-21kworker/u8:2+flush-259:022:45:181
4044911992117,3cyclictest4189795-21kworker/u8:1+events_unbound21:35:020
4044922992017,2cyclictest4143128-21kworker/u8:2+flush-259:021:25:003
4044922992016,3cyclictest4117844-21kworker/u8:1+flush-259:020:35:213
4044917992017,2cyclictest4189795-21kworker/u8:1+flush-259:021:30:002
4044917992017,2cyclictest4045083-21kworker/u8:4+events_unbound19:35:252
4044917992017,2cyclictest4045083-21kworker/u8:4+events_unbound19:35:252
4044917992017,2cyclictest4006506-21kworker/u8:2+flush-259:019:55:182
4044917992016,3cyclictest95595-21kworker/u8:3+flush-259:023:20:242
4044914992016,3cyclictest152745-21kworker/u8:2+flush-259:000:35:171
4044914992015,4cyclictest252-21systemd-journal00:35:001
4044911992016,3cyclictest152745-21kworker/u8:2+flush-259:000:05:220
4044911992016,3cyclictest152745-21kworker/u8:2+flush-259:000:05:220
4044911992014,4cyclictest509-21rs:main0
4044922991916,2cyclictest48438-21kworker/u8:2+flush-259:023:45:303
4044922991916,2cyclictest4006506-21kworker/u8:2+flush-259:019:40:133
4044922991916,2cyclictest152855-21idleruntime-cro00:00:003
4044922991915,3cyclictest4065972-21kworker/u8:3+events_unbound20:30:023
4044922991915,3cyclictest11250-21kworker/u8:3+events_unbound22:05:203
4044922991915,2cyclictest48438-21kworker/u8:2+events_unbound22:45:193
4044922991915,2cyclictest1412-21kworker/u8:0+flush-259:021:55:183
4044917991917,1cyclictest4168206-21ntpq21:05:212
4044917991916,2cyclictest4153833-21kworker/u8:1+flush-259:020:55:182
4044917991916,2cyclictest152745-21kworker/u8:2+flush-259:000:15:172
4044917991915,3cyclictest48438-21kworker/u8:2+flush-259:023:10:182
4044917991915,3cyclictest4065972-21kworker/u8:3+flush-259:020:30:012
4044917991915,3cyclictest152745-21kworker/u8:2+flush-259:000:25:202
4044917991913,4cyclictest252-21systemd-journal23:20:002
404491499192,8cyclictest4158800-21awk21:00:011
4044914991914,3cyclictest4006506-21kworker/u8:2+flush-259:019:55:261
4044914991914,3cyclictest22734-21/usr/sbin/munin21:55:231
4044911991916,2cyclictest53538-21kworker/u8:4+events_unbound22:50:150
4044911991916,2cyclictest4085551-21kworker/u8:1+flush-259:019:50:280
4044911991915,3cyclictest4091774-21kworker/u8:4+events_unbound20:29:570
4044911991915,3cyclictest152745-21kworker/u8:2+events_unbound00:35:160
4044922991815,2cyclictest1412-21kworker/u8:0+flush-259:022:20:133
4044922991814,3cyclictest53538-21kworker/u8:4+flush-259:022:55:233
4044922991814,3cyclictest4117844-21kworker/u8:1+events_unbound20:45:163
4044922991814,3cyclictest4085551-21kworker/u8:1+flush-259:019:50:153
4044917991815,2cyclictest48438-21kworker/u8:2+events_unbound23:30:222
4044917991815,2cyclictest4091774-21kworker/u8:4+flush-259:020:35:262
4044917991815,2cyclictest116875-21kworker/u8:1+flush-259:023:55:212
4044917991814,3cyclictest4034122-21kworker/u8:1+flush-259:019:10:252
4044917991814,3cyclictest127011-21kworker/u8:5+flush-259:023:50:202
4044917991814,3cyclictest110905-21kworker/u8:4+events_unbound23:25:182
404491799180,16cyclictest1-21systemd19:20:002
4044914991814,2cyclictest153142-21apt-get00:00:131
4044914991814,2cyclictest116875-21kworker/u8:1+flush-259:023:45:251
4044914991813,3cyclictest4117844-21kworker/u8:1+flush-259:020:40:161
4044914991810,5cyclictest4112450-21latency_hist20:15:001
4044911991815,2cyclictest152745-21kworker/u8:2+events_unbound00:15:240
4044911991815,2cyclictest116875-21kworker/u8:1+events_unbound23:45:300
4044911991814,3cyclictest4091774-21kworker/u8:4+flush-259:020:14:100
4044922991714,2cyclictest53538-21kworker/u8:4+events_unbound22:50:193
4044922991714,2cyclictest4143128-21kworker/u8:2+flush-259:021:25:203
4044922991713,3cyclictest48438-21kworker/u8:2+flush-259:022:40:163
4044922991713,3cyclictest48438-21kworker/u8:2+events_unbound23:10:233
4044922991713,3cyclictest110905-21kworker/u8:4+events_unbound23:30:163
4044917991714,2cyclictest48438-21kworker/u8:2+events_unbound22:45:172
4044917991714,2cyclictest4091774-21kworker/u8:4+flush-259:020:05:182
4044917991713,3cyclictest4065972-21kworker/u8:3+flush-259:020:20:222
4044914991713,2cyclictest0-21swapper/121:30:001
4044914991711,4cyclictest152868-21latency_hist00:00:001
4044911991714,2cyclictest60350irq/132-enp0s29f1:rx-021:05:160
4044911991714,2cyclictest4091774-21kworker/u8:4+events_unbound20:01:550
4044911991713,3cyclictest4034122-21kworker/u8:1+events_unbound19:10:290
4044911991711,4cyclictest4044828-21latency_hist19:10:000
4044922991613,2cyclictest4055832-21kworker/u8:0+flush-259:019:20:153
4044922991612,3cyclictest22514-21kworker/u8:1+flush-259:022:15:193
4044922991612,3cyclictest110905-21kworker/u8:4+flush-259:023:25:173
4044922991612,2cyclictest59015-21idleruntime-cro22:30:003
4044922991611,4cyclictest4143128-21kworker/u8:2+flush-259:021:10:243
4044922991611,4cyclictest252-21systemd-journal21:45:003
4044922991610,4cyclictest95574-21latency_hist23:05:013
4044917991613,2cyclictest1412-21kworker/u8:0+events_unbound22:20:232
4044917991612,3cyclictest4117844-21kworker/u8:1+flush-259:020:40:142
4044917991612,3cyclictest178979-21kworker/u8:0+flush-259:000:30:202
4044917991612,3cyclictest152745-21kworker/u8:2+events_unbound00:05:252
4044917991611,4cyclictest64851-21apt-get22:35:132
404491799160,14cyclictest59045-21unin-run22:30:002
4044911991614,1cyclictest60350irq/132-enp0s29f1:rx-021:55:170
4044911991614,1cyclictest60350irq/132-enp0s29f1:rx-020:50:130
4044911991613,2cyclictest75133-21kworker/u8:0+flush-259:023:15:250
4044911991612,3cyclictest158473-21kworker/u8:4+events_unbound00:10:270
4044922991511,3cyclictest4148481-21kworker/u8:3+flush-259:021:00:263
4044922991511,3cyclictest4091774-21kworker/u8:4+events_unbound20:40:243
4044922991511,3cyclictest4045083-21kworker/u8:4+flush-259:019:25:283
4044922991511,3cyclictest116875-21kworker/u8:1+events_unbound23:40:233
4044917991512,2cyclictest90050-21kworker/u8:1+events_unbound23:00:242
4044917991512,2cyclictest85447-21/usr/sbin/munin22:55:212
4044917991512,2cyclictest116875-21kworker/u8:1+events_unbound23:40:212
404491499158,5cyclictest252-21systemd-journal22:09:091
4044914991512,2cyclictest4091774-21kworker/u8:4+flush-259:020:25:221
4044914991512,2cyclictest1412-21kworker/u8:0+flush-259:022:00:241
4044914991512,2cyclictest0-21swapper/120:40:011
4044914991511,3cyclictest4006506-21kworker/u8:2+events_unbound19:40:191
4044914991511,3cyclictest110905-21kworker/u8:4+flush-259:023:20:271
4044914991511,2cyclictest0-21swapper/123:55:001
4044914991510,4cyclictest4174496-21/usr/sbin/munin21:15:171
4044911991513,1cyclictest60350irq/132-enp0s29f1:rx-023:05:270
4044911991513,1cyclictest60350irq/132-enp0s29f1:rx-020:16:320
4044911991512,2cyclictest60350irq/132-enp0s29f1:rx-023:10:470
4044922991411,2cyclictest64086-21kworker/u8:1+events_unbound22:35:163
4044922991411,2cyclictest4127932-21kworker/u8:0+flush-259:020:55:303
4044922991411,2cyclictest4124064-21kthreadcore20:25:223
4044922991411,2cyclictest4082132-21irqcore19:45:183
4044922991411,2cyclictest167581-21ntp_offset00:10:223
404491799149,4cyclictest1412-21kworker/u8:0+flush-259:021:45:102
4044917991412,1cyclictest0-21swapper/220:00:182
4044917991411,2cyclictest48438-21kworker/u8:2+flush-259:022:30:212
4044917991411,2cyclictest4091774-21kworker/u8:4+events_unbound20:30:152
4044917991411,2cyclictest110905-21kworker/u8:4+events_unbound23:35:262
404491499149,4cyclictest116875-21kworker/u8:1+flush-259:023:30:251
404491499148,4cyclictest4153603-21latency_hist20:55:011
404491499148,4cyclictest252-21systemd-journal22:40:001
404491499147,5cyclictest11742-21apt-get21:45:011
4044914991411,2cyclictest4082742-21kthreadcore19:45:171
4044914991411,2cyclictest4006506-21kworker/u8:2+flush-259:019:30:271
4044914991410,3cyclictest1390-21sh21:35:001
404492299139,3cyclictest4148481-21kworker/u8:3+events_unbound20:50:183
4044922991311,1cyclictest0-21swapper/319:14:593
4044922991310,2cyclictest4050425-21cpuspeed_turbos19:15:143
4044922991310,2cyclictest4006506-21kworker/u8:2+flush-259:019:55:163
404491799139,3cyclictest4143128-21kworker/u8:2+events_unbound21:10:142
4044917991311,1cyclictest22495-21turbostat.cron21:55:002
4044917991311,1cyclictest22495-21turbostat.cron21:55:002
4044917991310,2cyclictest191697-21kthreadcore00:35:172
4044917991310,2cyclictest11250-21kworker/u8:3+flush-259:022:15:212
404491499139,3cyclictest178734-21cron00:25:001
404491499138,4cyclictest4184670-21gpgv21:24:591
404491499137,4cyclictest252-21systemd-journal20:35:011
404491499136,5cyclictest4163112-21ntpq21:00:231
4044914991311,1cyclictest0-21swapper/122:25:001
404492299129,2cyclictest4143128-21kworker/u8:2+flush-259:021:35:003
404492299129,2cyclictest174253-21sed00:20:163
4044922991210,1cyclictest0-21swapper/320:05:173
404491799129,2cyclictest1412-21kworker/u8:0+events_unbound22:00:222
404491799121,2cyclictest85194-21awk22:55:002
4044917991210,1cyclictest64086-21kworker/u8:1+events_unbound22:40:212
404491499129,2cyclictest4096934-21kworker/u8:5+flush-259:020:16:411
404491499129,2cyclictest21483-21needreboot21:50:211
404491499129,2cyclictest21483-21needreboot21:50:211
404491499129,2cyclictest0-21swapper/121:15:001
404491499126,4cyclictest111224-21cp23:20:011
404492299118,2cyclictest4174128-21kworker/u8:3+flush-259:021:15:253
404492299117,3cyclictest75133-21kworker/u8:0+flush-259:023:21:453
404491799118,2cyclictest4145867-21kthreadcore20:45:192
404491799118,2cyclictest4065972-21kworker/u8:3+events_unbound20:10:232
404491799116,4cyclictest116875-21kworker/u8:1+events_unbound23:45:152
404491499119,1cyclictest0-21swapper/119:40:001
404491499119,1cyclictest0-21swapper/119:39:591
404491499117,3cyclictest84444-21grep22:50:251
404491499107,2cyclictest129581-21kthreadcore23:35:211
404491499106,3cyclictest12659-21grep21:45:141
404491499105,3cyclictest172935-21sed00:15:241
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional