You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-04 - 13:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Fri Sep 04, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
78902236161,26sleep00-21swapper/019:05:110
95442202165,24sleep30-21swapper/319:06:073
95522201164,24sleep10-21swapper/119:06:121
95482201164,25sleep20-21swapper/219:06:102
9941998373,9cyclictest0-21swapper/222:34:382
116662590,2sleep20-21swapper/222:50:282
23552580,4sleep011-21rcuc/022:30:180
163252540,3sleep10-21swapper/100:15:181
227142530,3sleep30-21swapper/323:15:253
25982460,1sleep10-21swapper/123:45:121
992999379,13cyclictest19382-21sadc19:30:000
205562360,4sleep0992999cyclictest19:30:190
994899341,23cyclictest0-21swapper/321:16:383
992999325,12cyclictest15622-21idleruntime-cro20:35:000
994899311,20cyclictest0-21swapper/320:23:073
994899310,11cyclictest0-21swapper/321:21:343
9929993112,16cyclictest11830-21munin-run21:40:010
994899300,17cyclictest0-21swapper/321:55:143
992999306,11cyclictest26595-40dnf21:00:020
992999306,11cyclictest26595-40dnf21:00:020
994899281,11cyclictest0-21swapper/321:03:233
994899280,11cyclictest0-21swapper/322:36:063
239372280,3sleep3994899cyclictest20:50:263
994899270,13cyclictest0-21swapper/323:33:433
9936992713,12cyclictest28918-21kworker/u8:019:11:591
994899260,10cyclictest0-21swapper/319:24:463
992999261,18cyclictest0-21swapper/021:10:510
994899251,9cyclictest0-21swapper/319:36:483
994899251,10cyclictest0-21swapper/321:48:543
994899251,10cyclictest0-21swapper/321:32:423
994899250,14cyclictest0-21swapper/320:45:033
994899250,11cyclictest0-21swapper/319:25:383
994899250,10cyclictest0-21swapper/320:25:313
993699254,15cyclictest0-21swapper/119:35:141
993699251,18cyclictest0-21swapper/119:23:511
993699251,18cyclictest0-21swapper/100:33:461
992999251,13cyclictest8055-21basename22:45:010
994899249,3cyclictest0-21swapper/319:12:093
994899246,2cyclictest0-21swapper/300:10:223
994899241,9cyclictest29718-21/usr/sbin/munin21:05:173
994899241,11cyclictest0-21swapper/322:45:483
994899240,10cyclictest14867-21/usr/sbin/munin23:00:253
994199240,19cyclictest0-21swapper/223:14:172
993699244,7cyclictest26595-40dnf21:00:021
9936992413,4cyclictest0-21swapper/122:30:321
994899231,9cyclictest14092-21ntp_states19:15:203
994899231,8cyclictest0-21swapper/323:56:543
994899231,8cyclictest0-21swapper/322:01:503
994899231,5cyclictest0-21swapper/323:14:143
994899231,11cyclictest0-21swapper/300:29:163
994899231,10cyclictest0-21swapper/319:47:183
994899231,10cyclictest0-21swapper/319:47:183
994899230,9cyclictest0-21swapper/321:54:223
994899230,12cyclictest0-21swapper/322:42:123
994899230,10cyclictest0-21swapper/322:25:023
993699232,15cyclictest0-21swapper/122:20:551
993699231,7cyclictest0-21swapper/100:12:231
993699231,17cyclictest0-21swapper/121:49:511
993699231,16cyclictest0-21swapper/123:04:021
993699230,16cyclictest0-21swapper/121:55:131
992999236,7cyclictest0-21swapper/021:30:440
992999236,2cyclictest87350irq/126-enp1s0-23:58:180
89532230,2sleep07-21ksoftirqd/000:00:130
994899228,3cyclictest30962-21ls23:35:193
994899228,3cyclictest25993-21if_enp1s022:10:203
994899226,9cyclictest0-21swapper/322:24:423
994899221,9cyclictest0-21swapper/300:00:303
994899221,13cyclictest0-21swapper/323:06:393
994899220,9cyclictest0-21swapper/323:29:053
994199226,3cyclictest0-21swapper/222:15:452
9936992216,3cyclictest9788-21latency00:00:181
993699221,5cyclictest0-21swapper/123:35:191
9936992214,4cyclictest20908-21/usr/sbin/munin20:45:161
9936992214,2cyclictest0-21swapper/122:59:081
9936992214,2cyclictest0-21swapper/121:42:061
993699221,16cyclictest0-21swapper/119:15:551
993699221,15cyclictest0-21swapper/123:56:471
993699221,15cyclictest0-21swapper/121:10:121
993699221,15cyclictest0-21swapper/120:51:141
993699221,15cyclictest0-21swapper/119:53:421
993699221,15cyclictest0-21swapper/119:44:331
993699221,15cyclictest0-21swapper/119:25:351
993699220,16cyclictest0-21swapper/123:54:371
992999227,5cyclictest0-21swapper/023:01:310
992999220,11cyclictest0-21swapper/022:03:420
994899218,3cyclictest0-21swapper/323:44:213
994899217,3cyclictest7389-21diskstats20:15:153
994899217,3cyclictest0-21swapper/322:15:483
994899216,3cyclictest0-21swapper/322:08:363
994899216,3cyclictest0-21swapper/322:08:363
994899216,3cyclictest0-21swapper/320:56:393
994899216,3cyclictest0-21swapper/320:56:393
994899216,3cyclictest0-21swapper/319:31:443
994899215,8cyclictest0-21swapper/320:32:513
994899214,3cyclictest0-21swapper/323:49:593
994899211,13cyclictest0-21swapper/321:10:373
994899211,12cyclictest0-21swapper/321:25:563
994899210,8cyclictest0-21swapper/300:33:223
994899210,7cyclictest0-21swapper/319:43:543
994199211,7cyclictest0-21swapper/221:23:472
994199211,6cyclictest0-21swapper/223:09:462
994199211,6cyclictest0-21swapper/221:51:112
9941992114,4cyclictest8474-21irqstats21:30:222
994199211,13cyclictest0-21swapper/200:34:092
993699219,3cyclictest0-21swapper/122:13:421
993699218,4cyclictest5005-21tr20:10:111
993699216,7cyclictest0-21swapper/122:42:511
993699216,13cyclictest5505-21kworker/u8:023:12:551
993699212,14cyclictest0-21swapper/122:05:141
993699212,14cyclictest0-21swapper/122:05:141
993699212,14cyclictest0-21swapper/120:16:271
993699211,6cyclictest0-21swapper/121:31:171
993699211,3cyclictest0-21swapper/122:25:251
993699211,15cyclictest0-21swapper/100:29:161
993699211,14cyclictest0-21swapper/120:02:481
993699211,11cyclictest10253-21perf22:50:021
993699210,15cyclictest0-21swapper/120:35:271
992999212,11cyclictest26825-21/usr/sbin/munin21:00:180
992999211,9cyclictest17744-21kworker/0:020:46:090
992999211,6cyclictest0-21swapper/022:39:020
992999211,3cyclictest25567-21swap00:35:260
992999211,11cyclictest4335-21ntp_states21:20:230
992999211,10cyclictest31068-21irqstats23:35:190
296422210,1sleep00-21swapper/022:20:120
129362211,2sleep20-21swapper/200:08:142
994899207,3cyclictest0-21swapper/320:01:023
994899205,3cyclictest0-21swapper/322:53:323
994899204,8cyclictest0-21swapper/320:05:273
994899204,3cyclictest0-21swapper/320:38:073
994899204,3cyclictest0-21swapper/320:11:453
994899202,10cyclictest24302-21/usr/sbin/munin00:35:203
994899201,8cyclictest11458-21diskstats00:05:143
994899201,13cyclictest0-21swapper/321:35:083
994899201,10cyclictest0-21swapper/319:59:253
994199201,8cyclictest790-21in:imjournal21:00:012
994199201,8cyclictest790-21in:imjournal21:00:012
994199201,6cyclictest0-21swapper/200:04:512
994199201,5cyclictest0-21swapper/221:11:452
9941992013,4cyclictest18017-21grep19:25:172
993699201,8cyclictest4170-21load21:20:191
9936992013,3cyclictest0-21swapper/122:18:091
9936992013,3cyclictest0-21swapper/121:36:171
993699201,17cyclictest0-21swapper/123:34:421
993699201,15cyclictest0-21swapper/123:16:051
993699201,14cyclictest0-21swapper/123:44:191
993699201,14cyclictest0-21swapper/123:27:231
993699201,14cyclictest0-21swapper/123:09:551
993699201,14cyclictest0-21swapper/121:52:591
993699201,14cyclictest0-21swapper/121:28:131
993699201,14cyclictest0-21swapper/121:06:001
993699201,14cyclictest0-21swapper/120:57:341
993699201,14cyclictest0-21swapper/120:57:341
993699201,14cyclictest0-21swapper/120:30:251
993699201,14cyclictest0-21swapper/120:21:221
993699201,14cyclictest0-21swapper/120:08:521
9936992010,2cyclictest0-21swapper/100:05:151
993699200,15cyclictest0-21swapper/119:57:351
992999207,3cyclictest0-21swapper/023:49:120
992999205,6cyclictest0-21swapper/022:16:160
992999203,5cyclictest8564-1kworker/u9:119:45:590
992999203,5cyclictest8564-1kworker/u9:119:45:590
992999202,11cyclictest4491-21/usr/sbin/munin23:50:240
992999202,10cyclictest25860-21/usr/sbin/munin23:25:160
992999201,4cyclictest0-21swapper/022:50:190
992999201,11cyclictest28405-21/usr/sbin/munin19:50:160
992999201,11cyclictest16916-21timerwakeupswit00:15:270
994899195,3cyclictest0-21swapper/320:41:193
994199195,6cyclictest30-21rcuc/221:05:252
994199191,8cyclictest0-21swapper/222:06:452
994199191,8cyclictest0-21swapper/222:06:452
994199191,7cyclictest0-21swapper/221:36:132
994199191,7cyclictest0-21swapper/200:27:542
994199191,6cyclictest0-21swapper/221:45:502
994199191,4cyclictest0-21swapper/222:58:292
993699195,4cyclictest0-21swapper/122:00:451
993699191,4cyclictest25253-21users23:20:261
9936991912,3cyclictest0-21swapper/121:19:051
9936991912,2cyclictest0-21swapper/120:41:131
993699191,16cyclictest0-21swapper/122:35:381
993699191,14cyclictest0-21swapper/119:46:271
993699191,14cyclictest0-21swapper/119:46:271
993699191,10cyclictest24648-21/usr/sbin/munin00:35:161
993699190,17cyclictest0-21swapper/119:30:121
993699190,14cyclictest0-21swapper/120:26:201
992999199,3cyclictest12058-21/usr/sbin/munin21:40:280
992999197,2cyclictest0-21swapper/021:28:520
992999195,4cyclictest0-21swapper/021:50:300
992999194,3cyclictest0-21swapper/022:58:240
992999194,3cyclictest0-21swapper/000:30:120
992999191,5cyclictest0-21swapper/022:29:070
992999191,12cyclictest0-21swapper/023:40:270
992999191,10cyclictest31001-21diskmemload21:16:200
992999191,10cyclictest29031-21/usr/sbin/munin21:05:270
992999190,6cyclictest0-21swapper/000:07:040
317782191,2sleep00-21swapper/019:55:250
994899186,3cyclictest0-21swapper/323:20:373
994899186,2cyclictest0-21swapper/300:20:223
994899186,2cyclictest0-21swapper/300:20:223
994899184,3cyclictest87650irq/129-enp1s0-21:40:203
994899181,7cyclictest0-21swapper/300:17:523
994899181,11cyclictest0-21swapper/322:33:003
994899180,2cyclictest0-21swapper/322:55:423
994899180,14cyclictest0-21swapper/323:50:173
994199181,7cyclictest7412-21cat21:29:562
994199181,7cyclictest0-21swapper/221:57:042
994199181,4cyclictest0-21swapper/223:19:082
994199181,14cyclictest29787-21grep22:20:132
994199181,11cyclictest0-21swapper/219:35:162
993699181,3cyclictest0-21swapper/122:52:581
993699181,16cyclictest0-21swapper/100:21:411
993699181,16cyclictest0-21swapper/100:21:401
992999187,3cyclictest0-21swapper/000:23:220
992999187,3cyclictest0-21swapper/000:23:220
992999186,3cyclictest18648-21/usr/sbin/munin21:55:190
992999186,3cyclictest0-21swapper/023:34:150
992999186,3cyclictest0-21swapper/000:12:250
992999185,3cyclictest0-21swapper/023:23:280
992999182,9cyclictest16179-21kworker/u8:223:06:110
992999181,9cyclictest0-21swapper/023:15:190
992999181,9cyclictest0-21swapper/023:10:290
992999181,9cyclictest0-21swapper/000:28:390
9929991812,4cyclictest12345-21df19:15:030
992999181,10cyclictest31001-21diskmemload21:46:170
257992181,3sleep20-21swapper/219:42:522
137232181,2sleep20-21swapper/221:41:172
130212181,3sleep20-21swapper/219:15:092
994199176,3cyclictest87550irq/128-enp1s0-20:10:262
994199174,5cyclictest30-21rcuc/223:35:142
994199172,5cyclictest11346-21processes19:10:212
994199172,12cyclictest5528-21tr23:50:232
994199171,9cyclictest0-21swapper/221:19:182
994199171,7cyclictest0-21swapper/223:22:352
994199171,6cyclictest24391-21latency_hist20:55:012
994199171,6cyclictest21127-21tail19:30:272
994199171,6cyclictest0-21swapper/223:44:212
994199171,6cyclictest0-21swapper/222:14:092
994199171,6cyclictest0-21swapper/219:45:572
994199171,6cyclictest0-21swapper/219:45:572
994199171,5cyclictest0-21swapper/219:50:142
994199171,5cyclictest0-21swapper/200:22:312
994199171,5cyclictest0-21swapper/200:22:312
994199171,4cyclictest0-21swapper/223:30:182
994199171,4cyclictest0-21swapper/223:00:172
994199171,4cyclictest0-21swapper/200:19:502
994199171,3cyclictest15938-21cpuspeed_turbos20:35:132
9941991711,4cyclictest10528-21cat20:20:252
9941991711,3cyclictest27846-21netstat21:00:212
9941991711,3cyclictest10240-21dump-pmu-power22:50:022
992999176,3cyclictest11441-21/usr/sbin/munin20:25:130
992999172,12cyclictest24394-21turbostat20:55:010
992999171,6cyclictest0-21swapper/022:09:140
992999171,6cyclictest0-21swapper/022:09:140
992999171,5cyclictest15825-21head19:20:170
992999171,3cyclictest25485-21expr22:10:160
9929991712,3cyclictest9648-21cat22:45:250
992999171,15cyclictest0-21swapper/020:21:030
9929991711,3cyclictest24072-21cpuspeed_turbos19:40:120
43972171,2sleep20-21swapper/220:07:242
20812170,2sleep20-21swapper/223:45:022
9948991610,3cyclictest29187-21cat19:50:183
994199165,3cyclictest87650irq/129-enp1s0-00:10:232
994199163,3cyclictest15980-21if_enp2s019:20:172
994199161,6cyclictest8193-21grep23:55:272
994199161,6cyclictest25860-21/usr/sbin/munin23:25:232
994199161,6cyclictest0-21swapper/220:25:522
9941991611,3cyclictest8399-21sed20:15:252
9941991611,3cyclictest3875-21/usr/sbin/munin22:35:172
9941991611,3cyclictest30604-21/usr/sbin/munin19:55:182
9941991611,3cyclictest21402-21processes20:45:222
9941991611,3cyclictest18026-21/usr/sbin/munin20:40:232
9941991610,3cyclictest827-21runrttasks22:40:072
9941991610,3cyclictest31842-21/usr/sbin/munin22:25:152
9941991610,3cyclictest24302-21/usr/sbin/munin00:35:182
994199160,7cyclictest14600-21sed20:30:182
994199160,6cyclictest0-21swapper/220:00:162
992999161,5cyclictest0-21swapper/019:10:120
992999161,13cyclictest21808-21/usr/sbin/munin19:35:240
9929991611,3cyclictest19543-21grep20:40:240
9929991610,3cyclictest333-21/usr/sbin/munin20:00:170
65802160,2sleep00-21swapper/020:12:100
225222160,3sleep20-21swapper/222:01:162
992999151,6cyclictest808-21snmpd20:15:030
9929991510,3cyclictest3715-21uname20:05:220
9929991510,3cyclictest15835-21/usr/sbin/munin20:35:250
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional