You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-23 - 05:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Wed Sep 23, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
205272239163,23sleep30-21swapper/319:08:213
187002239164,24sleep20-21swapper/219:05:102
206072204165,26sleep00-21swapper/019:09:230
204282203165,25sleep10-21swapper/119:07:021
20748991101,107cyclictest0-21swapper/221:31:572
20748996357,1cyclictest0-21swapper/223:24:562
30512530,1sleep00-21swapper/000:35:130
30512530,1sleep00-21swapper/000:35:130
126742470,2sleep20-21swapper/222:30:142
20748994643,2cyclictest0-21swapper/221:08:012
20748994537,3cyclictest0-21swapper/220:21:082
61462350,6sleep12074299cyclictest22:15:121
2074899350,26cyclictest0-21swapper/223:00:062
2073199357,13cyclictest30815-21munin-run20:45:010
2074899341,24cyclictest0-21swapper/200:02:462
2074899310,24cyclictest0-21swapper/223:26:142
2074899301,22cyclictest0-21swapper/220:49:302
2074899301,13cyclictest0-21swapper/219:18:302
2074899300,13cyclictest0-21swapper/223:10:542
2074899291,20cyclictest0-21swapper/221:02:222
2074899291,20cyclictest0-21swapper/221:02:222
2074899291,17cyclictest0-21swapper/222:18:102
2074899290,23cyclictest0-21swapper/200:36:582
2074899290,23cyclictest0-21swapper/200:36:582
2074899281,9cyclictest0-21swapper/221:14:462
2074899281,13cyclictest0-21swapper/222:08:262
2074899280,12cyclictest0-21swapper/222:00:262
2074299287,16cyclictest32483-40dnf00:27:241
2074299285,13cyclictest14695-21sed23:45:271
2073199283,11cyclictest15444-21perf20:10:010
2074899271,8cyclictest0-21swapper/221:25:142
2074899271,18cyclictest0-21swapper/220:52:302
2074899271,11cyclictest0-21swapper/220:13:542
2074899271,10cyclictest0-21swapper/220:32:582
2074899270,24cyclictest0-21swapper/200:17:022
2074899270,17cyclictest0-21swapper/223:17:462
2074899270,13cyclictest0-21swapper/219:39:342
2074899270,12cyclictest0-21swapper/221:21:222
2074899270,11cyclictest0-21swapper/200:08:342
2074899270,10cyclictest0-21swapper/223:56:222
2074899270,10cyclictest0-21swapper/219:32:462
259762260,2sleep011-21rcuc/023:00:140
226132260,2sleep10-21swapper/100:05:161
2074899261,9cyclictest0-21swapper/220:55:262
2074899261,11cyclictest0-21swapper/222:22:502
2074899260,15cyclictest0-21swapper/222:59:462
2074899260,12cyclictest0-21swapper/200:26:502
2074899260,10cyclictest0-21swapper/221:52:382
2074299260,18cyclictest32291-21/usr/sbin/munin23:15:131
2073199263,9cyclictest11-21rcuc/021:55:010
2073199261,12cyclictest16061-21awk21:25:010
2074899258,3cyclictest0-21swapper/222:14:402
2074899258,3cyclictest0-21swapper/222:14:402
2074899256,3cyclictest0-21swapper/221:48:582
2074899251,9cyclictest0-21swapper/219:21:502
2074899251,8cyclictest0-21swapper/221:18:422
2074899251,11cyclictest0-21swapper/200:11:582
2074899250,12cyclictest0-21swapper/219:40:022
2074899250,10cyclictest0-21swapper/222:47:462
2074899250,10cyclictest0-21swapper/222:28:302
2074299251,12cyclictest12282-21awk22:30:011
2074299251,11cyclictest0-21swapper/119:11:181
2074299250,6cyclictest0-21swapper/123:23:581
2074299250,23cyclictest0-21swapper/119:23:231
20731992510,9cyclictest0-21swapper/023:33:410
2075599248,4cyclictest0-21swapper/323:54:013
2075599242,11cyclictest11882-21/usr/sbin/munin21:15:273
2075599241,11cyclictest30731-21/usr/sbin/munin00:25:203
2074899241,9cyclictest0-21swapper/222:50:502
2074899240,9cyclictest0-21swapper/222:43:342
2074899240,9cyclictest0-21swapper/221:58:582
2074899240,9cyclictest0-21swapper/221:42:502
2074899240,10cyclictest0-21swapper/223:44:102
2074899240,10cyclictest0-21swapper/219:14:382
2074299244,6cyclictest29235-21dump-pmu-power21:55:011
2074299241,16cyclictest0-21swapper/123:44:131
2073199247,14cyclictest1916-21perf23:20:010
2073199241,4cyclictest0-21swapper/000:25:160
2075599238,3cyclictest0-21swapper/321:25:413
2075599231,3cyclictest0-21swapper/322:54:153
2075599231,11cyclictest3872-21cpuspeed_turbos22:10:183
2075599231,11cyclictest3872-21cpuspeed_turbos22:10:183
2075599231,11cyclictest21029-21sshd22:49:413
2075599231,10cyclictest0-21swapper/323:06:233
20755992310,2cyclictest2128-21/usr/sbin/munin23:20:153
2074899237,3cyclictest0-21swapper/221:39:502
2074899237,10cyclictest0-21swapper/220:41:062
2074899235,3cyclictest0-21swapper/223:30:382
2074899231,9cyclictest0-21swapper/219:49:542
2074899231,11cyclictest0-21swapper/222:37:022
2074899231,10cyclictest0-21swapper/223:45:422
2074899230,9cyclictest0-21swapper/223:50:262
2074899230,9cyclictest0-21swapper/220:05:222
2074299239,3cyclictest87650irq/129-enp1s0-23:55:131
2074299236,7cyclictest0-21swapper/122:14:461
2074299236,7cyclictest0-21swapper/122:14:461
2074299233,17cyclictest221ktimersoftd/100:35:171
2074299233,17cyclictest221ktimersoftd/100:35:171
2074299231,9cyclictest22392-21cpuspeed_turbos20:25:121
20742992314,3cyclictest0-21swapper/123:39:411
2074299231,19cyclictest0-21swapper/120:53:401
2074299231,16cyclictest0-21swapper/123:32:201
2073199239,6cyclictest0-21swapper/022:54:190
2073199231,16cyclictest0-21swapper/022:30:500
2073199230,16cyclictest0-21swapper/023:24:020
2073199230,15cyclictest0-21swapper/023:14:460
2075599221,7cyclictest0-21swapper/321:47:493
2075599221,11cyclictest87350irq/126-enp1s0-21:10:263
2075599221,10cyclictest26337-21/usr/sbin/munin00:15:243
2074899221,11cyclictest11600-21entropy20:00:142
2074899220,9cyclictest0-21swapper/219:50:222
2074899220,9cyclictest0-21swapper/219:50:222
2074299229,4cyclictest24179-21df_inode19:15:131
2074299224,7cyclictest5975-21runrttasks23:26:041
2074299223,16cyclictest29871-21awk23:10:011
2074299223,11cyclictest8444-21df22:20:191
2074299222,11cyclictest25421-21diskstats21:45:151
20742992213,3cyclictest0-21swapper/100:15:351
2074299221,2cyclictest261982sleep120:32:181
2074299221,16cyclictest0-21swapper/121:12:431
2074299221,15cyclictest0-21swapper/100:31:511
2074299221,10cyclictest0-21swapper/119:56:061
2074299220,20cyclictest0-21swapper/121:07:281
2073199226,7cyclictest0-21swapper/022:41:060
2073199221,15cyclictest0-21swapper/023:49:180
2073199221,14cyclictest0-21swapper/021:55:540
2073199221,14cyclictest0-21swapper/021:33:200
20731992211,3cyclictest0-21swapper/000:10:120
2073199220,15cyclictest0-21swapper/023:08:480
2075599217,3cyclictest7487-21/usr/sbin/munin21:05:273
2075599215,8cyclictest87350irq/126-enp1s0-22:35:253
2075599215,7cyclictest0-21swapper/321:20:253
2075599215,5cyclictest8595-21latency_hist23:35:013
2075599213,9cyclictest24282-21cpuspeed_turbos00:10:153
2075599213,4cyclictest23251-21needreboot20:25:193
2075599211,7cyclictest28808-21perl19:25:163
2075599211,7cyclictest27604-21df21:50:133
2075599211,6cyclictest25029-21perl20:30:163
2075599211,10cyclictest23496-21/usr/sbin/munin22:55:153
2075599210,12cyclictest0-21swapper/321:33:403
2074899215,5cyclictest0-21swapper/219:29:182
2074899210,8cyclictest0-21swapper/200:24:302
2074899210,11cyclictest0-21swapper/223:09:182
2074299216,4cyclictest0-21swapper/122:57:201
2074299215,9cyclictest0-21swapper/123:00:191
2074299212,10cyclictest1311-21cat22:05:011
2074299211,18cyclictest0-21swapper/120:01:391
2074299211,18cyclictest0-21swapper/119:30:151
2074299211,15cyclictest0-21swapper/122:07:421
2074299211,14cyclictest0-21swapper/100:03:291
2074299210,16cyclictest0-21swapper/123:50:151
2073199218,3cyclictest10021-21seq22:24:300
2073199211,15cyclictest0-21swapper/023:51:110
2073199211,15cyclictest0-21swapper/022:45:220
2073199211,15cyclictest0-21swapper/022:19:440
2073199211,14cyclictest0-21swapper/022:08:350
2073199211,13cyclictest87350irq/126-enp1s0-00:00:150
2073199211,12cyclictest0-21swapper/022:58:070
2073199211,10cyclictest13229-21crond20:05:010
20731992110,3cyclictest23835-21/usr/sbin/munin19:15:250
20731992110,3cyclictest0-21swapper/021:46:360
2075599208,10cyclictest18869-21awk22:45:013
2075599206,3cyclictest8564-1kworker/u9:121:56:403
2075599204,4cyclictest1467-21cat22:05:013
2075599203,8cyclictest0-21swapper/323:38:353
2075599202,9cyclictest755-21rngd00:39:063
2075599202,9cyclictest755-21rngd00:39:063
2075599202,5cyclictest879-21diskstats00:30:133
2075599201,9cyclictest5975-21kworker/u8:121:03:453
2075599201,9cyclictest5975-21kworker/u8:121:03:453
2075599201,9cyclictest2089-21kworker/u8:019:11:453
2075599201,8cyclictest22961-21cpuspeed_turbos21:40:113
2075599201,7cyclictest11264-21/usr/sbin/munin23:40:123
2075599201,7cyclictest0-21swapper/323:18:103
2075599201,7cyclictest0-21swapper/323:10:213
2075599201,6cyclictest24085-21cat00:10:023
2074899201,5cyclictest0-21swapper/200:34:352
2074899201,12cyclictest0-21swapper/223:39:262
2074299206,6cyclictest5975-21kworker/u8:120:49:111
2074299206,2cyclictest6231-21missed_timers21:00:191
2074299206,2cyclictest6231-21missed_timers21:00:191
2074299204,9cyclictest0-21swapper/122:37:551
2074299204,7cyclictest21-21rcuc/119:35:361
2074299204,7cyclictest0-21swapper/121:35:561
2074299203,9cyclictest23133-21taskset22:53:271
2074299202,5cyclictest827-21runrttasks21:31:211
2074299202,16cyclictest0-21swapper/120:35:501
2074299202,16cyclictest0-21swapper/120:09:161
2074299202,16cyclictest0-21swapper/119:45:081
2074299202,15cyclictest19583-21fschecks_time22:45:161
20742992016,3cyclictest0-21swapper/120:59:171
20742992015,3cyclictest0-21swapper/121:16:501
20742992015,3cyclictest0-21swapper/119:53:441
20742992015,3cyclictest0-21swapper/119:53:441
2074299201,3cyclictest0-21swapper/120:14:201
20742992013,2cyclictest0-21swapper/100:11:491
2074299201,18cyclictest0-21swapper/120:44:331
2074299201,11cyclictest0-21swapper/100:20:221
2074299201,10cyclictest19822-21ls20:20:011
2073199208,3cyclictest0-21swapper/000:32:090
2073199207,5cyclictest21718-21if_err_enp1s019:10:160
2073199202,9cyclictest10654-21vmstat19:55:270
2073199202,6cyclictest4327-21/usr/sbin/munin23:25:160
2073199202,5cyclictest4176-21kworker/0:119:45:010
2073199201,7cyclictest3714-21/usr/sbin/munin22:10:170
2073199201,7cyclictest3714-21/usr/sbin/munin22:10:170
20731992014,3cyclictest11573-21gltestperf23:40:140
2073199201,3cyclictest87350irq/126-enp1s0-21:35:040
2073199201,13cyclictest0-21swapper/023:38:340
2073199201,13cyclictest0-21swapper/022:28:010
2073199201,13cyclictest0-21swapper/021:09:400
2073199201,13cyclictest0-21swapper/000:17:250
2073199201,12cyclictest0-21swapper/000:23:590
2073199201,10cyclictest28009-21munin-run19:25:010
20731992010,2cyclictest0-21swapper/000:05:310
2073199200,5cyclictest0-21swapper/021:12:270
2073199200,4cyclictest0-21swapper/021:29:370
2073199200,12cyclictest0-21swapper/022:01:130
45322190,4sleep02073199cyclictest20:55:250
231222190,4sleep011-21rcuc/021:40:110
2075599194,3cyclictest9452-21diskmemload23:25:133
2075599193,5cyclictest39-21rcuc/300:00:183
2075599192,5cyclictest12506-21basename22:30:133
2075599191,9cyclictest0-21swapper/321:37:363
2075599191,6cyclictest11214-21memory22:25:193
2075599191,4cyclictest6437-21sshd19:47:403
2075599191,3cyclictest6968-21ntp_states22:15:213
20755991913,4cyclictest18532-21netstat23:55:203
20755991913,4cyclictest15430-21grep20:10:013
2075599191,15cyclictest21633-21grep20:20:273
2074299192,14cyclictest13415-21memory22:30:201
2074299192,11cyclictest9-21rcu_preempt21:20:141
20742991915,3cyclictest0-21swapper/122:40:591
20742991915,3cyclictest0-21swapper/121:26:011
20742991912,6cyclictest5975-21kworker/u8:120:20:381
2074299191,16cyclictest0-21swapper/119:25:071
2074299191,13cyclictest87650irq/129-enp1s0-21:44:491
2074299190,17cyclictest0-21swapper/119:40:161
20731991913,3cyclictest827-21runrttasks19:50:200
20731991913,3cyclictest827-21runrttasks19:50:200
2073199191,12cyclictest0-21swapper/023:55:110
2073199191,11cyclictest6513-21sshd19:48:400
2073199191,11cyclictest0-21swapper/022:35:180
2073199191,11cyclictest0-21swapper/021:15:020
2073199191,10cyclictest20967-21memory20:20:190
2075599183,3cyclictest0-21swapper/323:45:283
2075599182,4cyclictest25691-21/usr/sbin/munin23:00:143
2075599182,14cyclictest28842-21/usr/sbin/munin20:40:183
2075599181,9cyclictest0-21swapper/322:23:203
2075599181,9cyclictest0-21swapper/300:21:443
2075599181,5cyclictest17254-21vmstat20:10:263
2075599181,14cyclictest26030-21/usr/sbin/munin19:20:123
2075599180,7cyclictest0-21swapper/319:35:203
2075599180,10cyclictest20729-21cyclictest20:50:013
2075599180,10cyclictest0-21swapper/320:00:213
2074899181,7cyclictest0-21swapper/219:58:502
2074299182,13cyclictest30019-21perl21:55:151
2074299181,13cyclictest30757-21head23:10:211
2073199184,3cyclictest23470-40dnf20:25:230
2073199181,9cyclictest0-21swapper/020:15:140
20731991812,3cyclictest1670-21grep19:35:260
2073199181,11cyclictest0-21swapper/019:28:040
2073199181,10cyclictest15655-21/usr/sbin/munin20:10:180
2073199181,10cyclictest0-21swapper/021:04:130
2073199181,10cyclictest0-21swapper/021:04:130
2073199181,10cyclictest0-21swapper/020:45:290
2075599175,3cyclictest87350irq/126-enp1s0-19:55:203
2075599172,3cyclictest23209-21expr19:15:023
2075599171,4cyclictest3654-21cat22:10:013
2075599171,4cyclictest3077-21/usr/sbin/munin20:55:223
2075599171,4cyclictest0-21swapper/320:39:563
2075599171,3cyclictest836-21/usr/sbin/munin20:50:213
2074899171,7cyclictest27899-21sendmail_mailtr20:35:222
2074899171,11cyclictest0-21swapper/220:17:062
2073199171,4cyclictest31557-21pmu-power19:30:220
20731991712,3cyclictest27737-21open_inodes20:35:200
20731991712,3cyclictest25155-21idleruntime20:30:170
2075599161,6cyclictest0-21swapper/319:30:243
2075599161,14cyclictest0-21swapper/319:52:123
2075599161,14cyclictest0-21swapper/319:52:123
20755991610,3cyclictest3934-21sed19:40:253
20748991612,3cyclictest0-21swapper/220:28:572
2073199169,4cyclictest1856-21cat20:50:200
20755991510,3cyclictest18034-21cstates20:15:133
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional