You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-11-17 - 14:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3s.osadl.org (updated Tue Nov 17, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
154732241166,24sleep20-21swapper/219:05:102
173662240165,24sleep10-21swapper/119:09:101
172732203165,25sleep00-21swapper/019:07:570
172072202165,24sleep30-21swapper/319:07:083
3254221760,3sleep11750799cyclictest20:55:121
3254221760,3sleep11750799cyclictest20:55:121
2010421370,4sleep01750699cyclictest20:25:220
1300821110,4sleep01750699cyclictest22:35:240
222012950,2sleep01750699cyclictest19:15:280
205332720,3sleep20-21swapper/219:15:042
126782660,10sleep232-21ksoftirqd/223:50:152
17511996360,2cyclictest0-21swapper/200:02:222
269662560,3sleep10-21swapper/120:40:241
6572480,1sleep30-21swapper/323:23:443
149812480,1sleep30-21swapper/320:15:183
235752430,2sleep00-21swapper/023:00:210
1751999431,30cyclictest0-21swapper/323:28:473
1751199411,28cyclictest0-21swapper/222:39:232
17519994029,4cyclictest0-21swapper/300:00:263
1751999381,21cyclictest0-21swapper/300:32:023
17519993627,3cyclictest0-21swapper/321:19:583
1751999361,21cyclictest0-21swapper/323:18:553
170772360,1sleep20-21swapper/222:45:232
1751999351,22cyclictest0-21swapper/320:48:433
1751999341,26cyclictest0-21swapper/323:06:513
1751999341,26cyclictest0-21swapper/322:42:513
1751999341,16cyclictest0-21swapper/321:48:103
1751999331,24cyclictest0-21swapper/323:41:233
1751999331,15cyclictest11818-21taskset23:49:513
1751999331,14cyclictest0-21swapper/322:58:153
243112320,3sleep31751999cyclictest21:50:123
1751999321,20cyclictest0-21swapper/322:13:063
1751999321,19cyclictest0-21swapper/323:39:353
1751999321,19cyclictest0-21swapper/300:09:503
1751999321,13cyclictest0-21swapper/300:29:583
1750699324,13cyclictest29776-21idleruntime-cro20:50:010
1751999311,28cyclictest0-21swapper/321:38:263
1751999311,24cyclictest0-21swapper/300:17:263
1751999311,20cyclictest0-21swapper/322:29:473
1751999311,19cyclictest0-21swapper/321:44:543
1751999311,19cyclictest0-21swapper/321:34:103
1751999311,19cyclictest0-21swapper/319:38:583
1751999311,15cyclictest0-21swapper/322:21:543
1751999311,12cyclictest0-21swapper/322:18:543
1751199311,13cyclictest0-21swapper/223:23:362
1750699317,11cyclictest29471-21idleruntime-cro00:30:010
1750699311,14cyclictest10009-21munin-run20:05:010
1751999301,27cyclictest0-21swapper/319:16:343
1751999301,21cyclictest0-21swapper/320:27:273
1751999301,20cyclictest0-21swapper/323:54:143
1751999301,20cyclictest0-21swapper/322:30:193
1751999301,19cyclictest0-21swapper/322:54:193
1751999301,19cyclictest0-21swapper/321:59:343
1751999301,19cyclictest0-21swapper/319:41:543
1751999301,14cyclictest0-21swapper/322:37:273
1751999301,12cyclictest0-21swapper/300:10:503
1750799305,11cyclictest0-21swapper/100:25:141
1750799301,19cyclictest0-21swapper/121:45:411
1750699302,13cyclictest1200-21idleruntime-cro19:45:000
1751999291,19cyclictest0-21swapper/300:35:303
1751999291,14cyclictest0-21swapper/323:58:543
1751999291,14cyclictest0-21swapper/322:47:593
1751999291,13cyclictest0-21swapper/322:01:383
17511992911,4cyclictest0-21swapper/223:39:502
1750699296,10cyclictest26638-21perf23:10:000
1751999281,19cyclictest0-21swapper/321:21:343
1751999281,19cyclictest0-21swapper/300:24:543
1751999281,19cyclictest0-21swapper/300:24:543
1751999281,11cyclictest0-21swapper/321:12:143
1751999280,11cyclictest0-21swapper/320:59:133
1751999280,11cyclictest0-21swapper/320:59:133
1751199281,13cyclictest23904-21irqstats00:15:162
1751199281,12cyclictest0-21swapper/222:56:202
17519992717,3cyclictest0-21swapper/320:52:513
17519992714,8cyclictest0-21swapper/319:30:223
1751999271,13cyclictest0-21swapper/319:52:513
1751999271,11cyclictest25239-21taskset20:38:173
1751999271,11cyclictest0-21swapper/323:01:513
1751999270,11cyclictest0-21swapper/321:08:323
1751199271,11cyclictest736-21diskstats22:10:132
1751199271,11cyclictest0-21swapper/221:51:222
1751199271,11cyclictest0-21swapper/221:46:462
17507992713,12cyclictest23-21ksoftirqd/119:42:231
1750699271,13cyclictest32601-21idleruntime-cro22:10:010
1751999261,2cyclictest0-21swapper/320:06:113
1751999261,18cyclictest0-21swapper/320:44:393
1751999261,15cyclictest0-21swapper/319:10:173
1751999261,13cyclictest0-21swapper/321:00:383
1751999261,13cyclictest0-21swapper/319:47:073
1751999261,13cyclictest0-21swapper/319:47:063
1751999261,12cyclictest0-21swapper/323:33:233
1751999261,12cyclictest0-21swapper/322:07:463
1751999261,12cyclictest0-21swapper/319:21:423
1751999261,10cyclictest0-21swapper/319:25:463
17511992613,3cyclictest0-21swapper/200:05:182
1751199261,23cyclictest0-21swapper/220:42:462
1751199261,13cyclictest0-21swapper/220:24:092
1751199261,12cyclictest0-21swapper/221:16:362
1751199261,11cyclictest2347-21cstates21:00:122
1751199261,11cyclictest0-21swapper/222:16:572
1751199261,10cyclictest0-21swapper/219:24:322
1750699269,14cyclictest1835-21perf21:00:020
1750699269,14cyclictest1835-21perf21:00:020
1751999251,11cyclictest0-21swapper/320:33:233
1751999250,9cyclictest0-21swapper/323:10:253
1751999250,9cyclictest0-21swapper/323:10:253
1751999250,11cyclictest0-21swapper/320:20:133
17511992514,3cyclictest0-21swapper/221:24:022
1751199251,13cyclictest0-21swapper/200:23:402
1751199251,13cyclictest0-21swapper/200:23:402
1751199251,12cyclictest31032-21fschecks_count22:05:152
1751199251,12cyclictest0-21swapper/222:43:372
1751199251,11cyclictest0-21swapper/223:11:222
1751199251,11cyclictest0-21swapper/223:11:222
1751199251,11cyclictest0-21swapper/221:34:282
17507992518,4cyclictest25184-21sshd20:37:341
1750799251,19cyclictest0-21swapper/122:13:401
1750699251,11cyclictest19417-21munin-run21:40:010
1751999241,13cyclictest0-21swapper/320:01:433
1751199249,3cyclictest0-21swapper/220:19:182
1751199241,9cyclictest0-21swapper/222:01:422
17511992412,8cyclictest0-21swapper/223:43:222
1751199241,20cyclictest0-21swapper/221:41:262
1750799241,4cyclictest21617-21perf21:45:011
1750799241,18cyclictest0-21swapper/122:44:231
1750699241,6cyclictest0-21swapper/023:40:120
1751999237,3cyclictest0-21swapper/321:25:423
1751999231,20cyclictest0-21swapper/319:57:393
1751999231,19cyclictest143052sleep320:13:593
1751199233,3cyclictest0-21swapper/220:35:182
1750799238,4cyclictest127902basename23:50:161
1750799238,12cyclictest28540-21grep22:00:151
1750799232,16cyclictest0-21swapper/121:09:551
17507992314,5cyclictest0-21swapper/100:22:561
17507992314,5cyclictest0-21swapper/100:22:561
1750799231,19cyclictest2723-21hddtemp_smartct21:00:151
1750799230,17cyclictest0-21swapper/122:05:261
1750699234,16cyclictest3403-21munin-run19:50:010
1750699234,16cyclictest3403-21munin-run19:50:010
1750699231,8cyclictest21310-21kworker/0:121:50:370
17511992214,3cyclictest0-21swapper/223:32:422
1751199221,15cyclictest28155-21df_inode20:45:142
17511992210,3cyclictest0-21swapper/222:25:152
1751199220,14cyclictest0-21swapper/223:55:182
1750799223,2cyclictest87550irq/128-enp1s0-22:31:091
1750799222,17cyclictest28825-21munin-run23:15:011
1750799222,17cyclictest28825-21munin-run23:15:011
1750799222,15cyclictest0-21swapper/123:04:441
17507992217,3cyclictest0-21swapper/123:39:011
17507992215,2cyclictest0-21swapper/119:56:511
17507992214,3cyclictest0-21swapper/121:18:501
17507992214,3cyclictest0-21swapper/100:05:171
17507992214,2cyclictest0-21swapper/123:18:431
1750799221,15cyclictest0-21swapper/123:45:441
1750799221,15cyclictest0-21swapper/122:20:061
1750799221,12cyclictest31549-21unixbench_singl20:50:231
1750699224,15cyclictest5266-21perf23:35:020
1750699221,9cyclictest32460-21kworker/0:022:18:140
1751199214,3cyclictest0-21swapper/220:08:232
1751199211,13cyclictest0-21swapper/220:29:012
1751199211,11cyclictest0-21swapper/200:31:222
1751199211,11cyclictest0-21swapper/200:25:142
1751199210,9cyclictest0-21swapper/221:29:222
1751199210,10cyclictest0-21swapper/220:58:462
1751199210,10cyclictest0-21swapper/220:58:462
1750799215,4cyclictest0-21swapper/123:05:311
1750799212,14cyclictest0-21swapper/122:59:161
1750799211,9cyclictest0-21swapper/122:52:021
17507992116,3cyclictest18456-21idleruntime-cro00:05:001
1750799211,3cyclictest0-21swapper/122:46:581
17507992113,3cyclictest0-21swapper/119:50:201
17507992113,3cyclictest0-21swapper/119:27:011
1750799211,18cyclictest0-21swapper/122:25:141
1750799211,18cyclictest0-21swapper/100:32:391
1750799211,15cyclictest0-21swapper/123:28:541
1750799211,15cyclictest0-21swapper/123:21:141
1750799211,15cyclictest0-21swapper/121:12:501
1750799211,15cyclictest0-21swapper/120:26:201
1750799211,15cyclictest0-21swapper/119:37:331
1750799211,14cyclictest87650irq/129-enp1s0-00:35:321
1750799211,14cyclictest0-21swapper/119:15:141
1750799211,11cyclictest14603-21df23:55:141
1750699212,9cyclictest28568-21kworker/0:323:14:380
1750699212,9cyclictest28568-21kworker/0:323:14:380
1750699212,11cyclictest29049-21/usr/sbin/munin23:15:190
1750699212,10cyclictest16191-21df_inode22:45:120
1750699211,11cyclictest10942-21ntp_states23:45:220
1751199208,3cyclictest0-21swapper/200:10:022
1751199206,3cyclictest0-21swapper/223:00:422
1751199204,3cyclictest0-21swapper/219:59:232
1751199201,13cyclictest0-21swapper/221:35:142
1751199201,12cyclictest0-21swapper/219:33:272
1751199201,11cyclictest0-21swapper/219:49:272
1751199201,11cyclictest0-21swapper/219:49:272
1750799202,17cyclictest0-21swapper/120:08:051
17507992016,3cyclictest0-21swapper/121:55:061
17507992013,2cyclictest0-21swapper/121:27:161
17507992013,2cyclictest0-21swapper/120:04:531
17507992012,3cyclictest0-21swapper/121:54:441
17507992012,3cyclictest0-21swapper/119:21:411
1750799201,14cyclictest0-21swapper/123:41:371
1750799201,14cyclictest0-21swapper/123:32:101
1750799201,14cyclictest0-21swapper/122:38:231
1750799201,14cyclictest0-21swapper/121:35:461
1750799201,14cyclictest0-21swapper/121:31:101
1750799201,14cyclictest0-21swapper/120:30:541
1750799201,14cyclictest0-21swapper/120:21:211
1750799201,14cyclictest0-21swapper/119:32:431
1750799201,12cyclictest0-21swapper/100:17:351
1750799201,11cyclictest3019-21diskstats22:15:131
1750799201,10cyclictest3409-21perf19:50:011
1750799201,10cyclictest3409-21perf19:50:011
1750799201,10cyclictest19769-21cat19:15:011
1750799200,15cyclictest0-21swapper/120:12:531
1750699209,3cyclictest5829-21/usr/sbin/munin19:55:150
1750699209,3cyclictest4867-21/usr/sbin/munin22:20:170
1750699207,4cyclictest1-21systemd20:10:180
1750699206,5cyclictest0-21swapper/000:34:210
1750699206,2cyclictest87350irq/126-enp1s0-22:11:520
1750699202,8cyclictest24207-21kworker/0:123:23:260
17506992014,4cyclictest14930-21cut23:55:170
1750699201,11cyclictest0-21swapper/022:56:590
1750699201,11cyclictest0-21swapper/000:37:110
1750699201,11cyclictest0-21swapper/000:12:320
1750699201,10cyclictest27386-21sshd20:42:340
108422201,15sleep111024-21cstates21:20:101
1751199199,3cyclictest0-21swapper/220:10:152
1751199195,2cyclictest0-21swapper/221:55:382
1751199194,7cyclictest87650irq/129-enp1s0-23:46:422
1751199193,13cyclictest17846-21tr19:10:132
1751199191,5cyclictest7220-21if_err_enp1s021:10:172
1751199191,5cyclictest29963-21memory23:15:182
1751199191,14cyclictest0-21swapper/222:31:492
1751199191,13cyclictest0-21swapper/222:52:052
1751199191,12cyclictest0-21swapper/219:35:172
1750799194,6cyclictest0-21swapper/100:10:101
17507991912,2cyclictest0-21swapper/120:46:561
1750699198,3cyclictest27201-21/usr/sbin/munin19:30:150
1750699195,5cyclictest0-21swapper/023:39:440
1750699194,6cyclictest0-21swapper/022:50:070
1750699193,5cyclictest32393-1kworker/u9:000:06:500
1750699192,12cyclictest0-21swapper/021:34:320
1750699191,9cyclictest0-21swapper/000:18:350
1750699191,8cyclictest22739-21kworker/0:200:21:140
1750699191,8cyclictest22739-21kworker/0:200:21:140
1750699191,6cyclictest0-21swapper/022:33:080
1750699191,6cyclictest0-21swapper/021:42:450
17506991912,4cyclictest1897-21irqstats23:25:180
1750699191,11cyclictest0-21swapper/021:26:010
1750699191,11cyclictest0-21swapper/021:14:430
1750699191,10cyclictest26230-21/usr/sbin/munin21:55:170
1750699191,10cyclictest23419-21/usr/sbin/munin20:35:180
1750699191,10cyclictest14631-21/usr/sbin/munin20:15:180
1750699191,10cyclictest12071-21/usr/sbin/munin23:50:150
1750699191,10cyclictest0-21swapper/022:00:440
1751199187,3cyclictest0-21swapper/220:31:262
1751199183,8cyclictest87550irq/128-enp1s0-20:01:312
1751199183,12cyclictest25166-21tr19:25:132
1751199181,3cyclictest0-21swapper/200:35:252
17511991812,3cyclictest0-21swapper/219:40:112
1750799181,2cyclictest0-21swapper/120:16:431
1750699185,4cyclictest0-21swapper/000:02:420
1750699183,3cyclictest0-21swapper/021:18:200
1750699182,3cyclictest7401-21df22:25:120
1750699181,5cyclictest0-21swapper/019:10:200
1750699181,2cyclictest87650irq/129-enp1s0-21:46:240
1750699181,10cyclictest0-21swapper/021:23:470
1751199175,3cyclictest0-21swapper/223:05:502
1751199175,1cyclictest87650irq/129-enp1s0-19:51:112
1751199172,6cyclictest0-21swapper/223:28:562
17511991712,3cyclictest30950-21missed_timers20:50:162
17511991710,4cyclictest0-21swapper/221:05:232
1750699173,3cyclictest0-21swapper/022:42:310
1750699171,6cyclictest0-21swapper/020:30:140
1750699171,5cyclictest32085-21cat20:55:020
17506991711,3cyclictest808-21snmpd20:20:420
1750699169,4cyclictest23466-21tail19:20:150
1750699161,6cyclictest0-21swapper/021:00:170
1750699161,14cyclictest0-21swapper/019:53:230
17506991611,3cyclictest29393-21/usr/sbin/munin19:35:130
17506991611,3cyclictest25099-21cpuspeed_turbos19:25:120
17506991610,4cyclictest6216-21crond21:10:020
17506991610,3cyclictest10233-21/usr/sbin/munin20:05:170
1751199151,3cyclictest87350irq/126-enp1s0-22:20:072
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional