You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-11 - 06:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Thu Jun 11, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
71212213184,19sleep30-21swapper/319:09:493
70822204167,24sleep10-21swapper/119:09:201
67852202163,26sleep20-21swapper/219:05:302
69792201163,26sleep00-21swapper/019:07:580
3098621480,5sleep3724199cyclictest22:28:383
3098621480,5sleep3724199cyclictest22:28:383
7236999693,2cyclictest0-21swapper/222:32:492
7236996966,2cyclictest0-21swapper/200:39:052
232602640,5sleep3724199cyclictest23:25:133
7236995249,2cyclictest0-21swapper/219:10:052
71512500,4sleep121-21rcuc/100:00:191
147102490,1sleep014711-21cat21:50:190
249592470,1sleep30-21swapper/319:45:243
319422460,2sleep30-21swapper/322:30:153
722599378,13cyclictest17910-21munin-run22:00:020
7225993711,12cyclictest6302-21munin-run20:20:000
21622350,2sleep20-21swapper/222:35:202
723099331,11cyclictest0-21swapper/122:54:371
723099321,12cyclictest0-21swapper/122:00:571
722599328,11cyclictest27652-21idleruntime-cro19:55:010
723099311,6cyclictest0-21swapper/119:15:371
723099311,13cyclictest0-21swapper/100:32:451
722599318,11cyclictest22326-21perf22:10:010
722599312,14cyclictest4106-21munin-run20:15:010
723099301,13cyclictest0-21swapper/122:35:231
723099301,13cyclictest0-21swapper/121:30:291
723099301,12cyclictest0-21swapper/123:28:361
723099301,11cyclictest0-21swapper/122:24:051
723099291,9cyclictest0-21swapper/120:32:391
723099291,12cyclictest0-21swapper/123:57:201
723099291,12cyclictest0-21swapper/121:29:531
723099291,11cyclictest0-21swapper/123:31:481
723099291,10cyclictest0-21swapper/121:02:191
723099291,10cyclictest0-21swapper/119:10:031
722599297,10cyclictest0-21swapper/022:30:000
722599297,10cyclictest0-21swapper/022:30:000
7225992910,16cyclictest29865-21munin-run20:00:010
723099281,9cyclictest0-21swapper/121:20:491
723099281,24cyclictest10878-21memory22:55:191
723099281,13cyclictest0-21swapper/120:07:451
723099281,13cyclictest0-21swapper/120:07:451
723099281,11cyclictest0-21swapper/119:32:491
723099281,11cyclictest0-21swapper/100:16:011
723099281,10cyclictest0-21swapper/119:23:221
723099280,11cyclictest0-21swapper/120:47:261
722599280,13cyclictest4736-21idleruntime-cro21:30:010
724199275,8cyclictest0-21swapper/300:26:173
723099278,7cyclictest31522-21irqstats21:15:171
723099278,7cyclictest31522-21irqstats21:15:171
723099271,9cyclictest22489-21/usr/sbin/munin20:55:141
723099271,8cyclictest0-21swapper/123:04:511
723099271,14cyclictest14712-21df_inode23:05:111
723099271,12cyclictest0-21swapper/122:16:371
723099271,12cyclictest0-21swapper/120:29:251
723099271,11cyclictest0-21swapper/100:20:571
723099271,11cyclictest0-21swapper/100:06:171
723099271,10cyclictest0-21swapper/121:10:571
723099271,10cyclictest0-21swapper/120:04:351
723099270,10cyclictest0-21swapper/121:39:471
722599271,12cyclictest31732-21awk23:45:010
723099269,7cyclictest0-21swapper/121:48:511
723099262,12cyclictest3373-21users23:50:241
723099261,9cyclictest0-21swapper/123:11:391
723099261,8cyclictest29233-21expr22:25:111
723099261,8cyclictest29233-21expr22:25:111
723099261,8cyclictest0-21swapper/123:18:551
723099261,16cyclictest0-21swapper/100:11:131
723099261,11cyclictest0-21swapper/123:40:521
723099260,11cyclictest0-21swapper/121:05:031
723099251,9cyclictest0-21swapper/122:47:171
723099251,9cyclictest0-21swapper/122:33:011
723099251,8cyclictest0-21swapper/120:42:081
7230992512,5cyclictest27338-21systemd-journal19:51:021
723099251,21cyclictest15713-21/usr/sbin/munin21:55:021
723099251,11cyclictest0-21swapper/121:42:311
723099251,11cyclictest0-21swapper/120:52:521
723099251,10cyclictest28285-21diskmemload00:26:371
723099251,10cyclictest0-21swapper/123:39:491
723099251,10cyclictest0-21swapper/123:39:481
723099251,10cyclictest0-21swapper/122:05:251
723099251,10cyclictest0-21swapper/119:44:491
723099250,10cyclictest0-21swapper/119:59:391
722599251,8cyclictest0-21swapper/022:23:210
7225992510,14cyclictest0-21swapper/000:05:020
724199246,10cyclictest0-21swapper/322:36:093
724199245,6cyclictest8564-1kworker/u9:123:00:203
7236992419,3cyclictest23882-21tune2fs19:45:142
723699241,5cyclictest1469-21sed20:05:252
723699241,5cyclictest1469-21sed20:05:252
723699241,11cyclictest0-21swapper/222:21:452
723099241,9cyclictest0-21swapper/120:24:151
723099241,9cyclictest0-21swapper/120:16:471
723099241,8cyclictest0-21swapper/119:28:151
723099241,10cyclictest0-21swapper/123:47:201
723099241,10cyclictest0-21swapper/122:13:431
723099241,10cyclictest0-21swapper/100:37:011
722599247,15cyclictest20131-21awk22:05:010
723699231,6cyclictest0-21swapper/222:53:462
7236992314,7cyclictest28285-21diskmemload22:28:382
7236992314,7cyclictest28285-21diskmemload22:28:382
7236992312,2cyclictest4720-21awk21:30:002
723099231,9cyclictest0-21swapper/120:39:581
722599231,3cyclictest87550irq/128-enp1s0-22:15:240
722599231,11cyclictest5966-21seq00:00:010
86402221,3sleep08744-21needreboot22:50:190
724199223,12cyclictest0-21swapper/323:41:133
7241992213,2cyclictest87550irq/128-enp1s0-23:07:413
724199221,11cyclictest26004-21head23:30:153
723699222,8cyclictest790-21in:imjournal20:56:412
723699221,8cyclictest0-21swapper/220:40:212
7236992215,4cyclictest28520-21gltestperf19:55:162
723099226,3cyclictest0-21swapper/119:45:071
723099224,10cyclictest28285-21diskmemload22:40:171
723099221,4cyclictest16731-21/usr/sbin/munin21:55:171
722599223,16cyclictest12589-21munin-run00:15:020
722599223,10cyclictest14369-21/usr/sbin/munin23:05:110
7225992217,3cyclictest11-21rcuc/019:15:020
31102220,1sleep30-21swapper/321:25:143
724199219,3cyclictest827-21runrttasks00:00:013
724199217,8cyclictest87350irq/126-enp1s0-00:20:183
724199216,7cyclictest0-21swapper/323:45:093
724199215,11cyclictest8825-21kworker/u8:023:22:353
724199211,8cyclictest1694-21pmu-power21:20:223
723699216,5cyclictest29060-21grep23:35:262
723699216,5cyclictest29060-21grep23:35:262
723699211,4cyclictest2524-21pluginstate21:25:002
723699211,2cyclictest87550irq/128-enp1s0-22:42:152
723699211,17cyclictest31645-21unixbench_singl20:00:272
723699211,11cyclictest0-21swapper/200:09:272
723099217,3cyclictest0-21swapper/120:14:251
722599216,9cyclictest7-21ksoftirqd/020:20:230
722599215,14cyclictest13531-21perf21:50:020
722599211,11cyclictest556-21ntp_states23:45:220
724199208,3cyclictest0-21swapper/323:38:133
724199208,3cyclictest0-21swapper/323:38:133
724199207,3cyclictest0-21swapper/321:00:093
724199206,6cyclictest15514-21sshd21:52:363
724199206,4cyclictest24822-21ls22:15:133
724199206,3cyclictest0-21swapper/320:58:573
724199202,7cyclictest909-21sssd_nss00:10:183
724199201,7cyclictest7728-21/usr/sbin/munin21:35:153
724199201,5cyclictest10452-21grep19:15:123
724199201,3cyclictest0-21swapper/322:20:193
724199201,16cyclictest8026-21grep19:10:143
7241992011,2cyclictest87550irq/128-enp1s0-22:56:503
724199200,10cyclictest28285-21diskmemload21:56:523
723699205,3cyclictest1863-21cpuspeed_turbos23:50:122
723699203,5cyclictest28634-21grep21:10:112
723699202,6cyclictest24602-21cat22:15:012
723699201,6cyclictest0-21swapper/223:43:432
723699201,6cyclictest0-21swapper/222:17:512
723699201,3cyclictest0-21swapper/223:07:412
723699200,2cyclictest0-21swapper/223:00:202
723099201,14cyclictest0-21swapper/119:35:431
723099200,12cyclictest0-21swapper/123:20:161
722599207,4cyclictest7-21ksoftirqd/000:37:230
722599201,7cyclictest1483-21df22:35:120
7225992015,3cyclictest20628-21find00:30:220
7225992013,4cyclictest1518-21grep21:20:190
722599201,13cyclictest0-21swapper/020:55:220
722599201,13cyclictest0-21swapper/019:33:450
722599201,11cyclictest808-21snmpd22:45:290
722599201,11cyclictest27548-21/usr/sbin/munin23:35:240
722599201,11cyclictest27548-21/usr/sbin/munin23:35:240
722599201,11cyclictest1784-21/usr/sbin/munin23:50:230
724199199,7cyclictest21329-21sshd20:50:373
724199196,3cyclictest0-21swapper/320:21:093
724199195,8cyclictest0-21swapper/322:51:163
724199195,4cyclictest9144-21cat00:05:163
724199195,4cyclictest1784-21/usr/sbin/munin23:50:183
724199194,6cyclictest0-21swapper/300:33:213
724199193,9cyclictest28285-21diskmemload23:13:143
724199193,9cyclictest28285-21diskmemload21:11:083
724199193,8cyclictest32064-21sh20:05:003
724199193,11cyclictest10141-21kworker/u8:221:40:493
724199191,9cyclictest5927-21/usr/sbin/munin22:45:143
724199191,9cyclictest0-21swapper/323:19:333
7241991914,3cyclictest15614-21ls20:40:133
724199191,12cyclictest0-21swapper/321:18:073
724199191,12cyclictest0-21swapper/321:18:073
724199191,10cyclictest4327-21/usr/sbin/munin20:15:263
7241991910,2cyclictest6912-21turbostat21:35:003
724199190,9cyclictest32392-21kworker/u8:122:10:023
724199190,17cyclictest0-21swapper/319:35:533
723699196,4cyclictest0-21swapper/223:34:392
723699195,9cyclictest5769-40dnf23:57:532
723699194,6cyclictest10705-21grep19:15:152
723699194,6cyclictest0-21swapper/200:15:472
723699194,4cyclictest0-21swapper/221:57:362
723699192,7cyclictest13515-21munin-run21:50:022
723699192,14cyclictest19701-21grep19:35:142
723699191,7cyclictest0-21swapper/200:21:212
723699191,6cyclictest10879-21memory22:55:192
723699191,5cyclictest9109-21idleruntime-cro21:40:012
723699191,5cyclictest18866-21crond19:35:012
723699191,5cyclictest18546-21fschecks_count22:00:152
723699191,10cyclictest0-21swapper/200:10:102
723699190,5cyclictest5024-21kworker/u8:123:29:002
722599194,14cyclictest0-21swapper/021:10:010
722599193,4cyclictest81ktimersoftd/021:15:010
722599191,6cyclictest0-21swapper/000:23:520
722599191,5cyclictest0-21swapper/022:40:410
722599191,10cyclictest10775-21irqstats22:55:170
724199186,3cyclictest87350irq/126-enp1s0-00:00:123
724199185,4cyclictest27930-21sshd21:05:373
724199185,3cyclictest8896-21cstates20:25:123
724199184,7cyclictest21294-21/usr/sbin/munin19:40:233
724199182,9cyclictest1908-21awk20:10:003
724199182,9cyclictest1908-21awk20:10:003
724199181,9cyclictest13121-21/usr/sbin/munin20:35:123
724199181,3cyclictest0-21swapper/322:05:523
724199181,12cyclictest0-21swapper/322:41:393
724199181,10cyclictest0-21swapper/321:45:473
724199180,15cyclictest4124-21cat20:15:013
724199180,13cyclictest0-21swapper/300:18:083
724199180,11cyclictest0-21swapper/322:00:163
723699185,4cyclictest22217-21sensors_temp23:20:232
723699185,3cyclictest87350irq/126-enp1s0-00:25:092
723699183,5cyclictest0-21swapper/200:00:212
723699183,4cyclictest32004-21switchtime21:15:232
723699183,4cyclictest32004-21switchtime21:15:232
723699183,2cyclictest87550irq/128-enp1s0-22:06:302
723699181,7cyclictest0-21swapper/222:46:172
723699181,6cyclictest6522-21/usr/sbin/munin20:20:112
723699181,5cyclictest10695-21sh20:30:012
722599189,2cyclictest808-21snmpd00:16:180
722599184,3cyclictest0-21swapper/021:31:010
722599181,6cyclictest0-21swapper/023:15:040
722599181,4cyclictest20948-21sshd19:38:380
722599181,14cyclictest22538-21/usr/sbin/munin22:10:160
722599180,12cyclictest0-21swapper/019:26:430
142052181,2sleep20-21swapper/219:21:372
111992181,2sleep20-21swapper/221:43:402
724199174,6cyclictest0-21swapper/300:35:113
724199171,3cyclictest0-21swapper/319:56:223
7241991713,3cyclictest0-21swapper/319:25:333
724199170,2cyclictest0-21swapper/319:50:153
723699174,3cyclictest87350irq/126-enp1s0-21:00:122
723699173,6cyclictest0-21swapper/223:10:122
723699172,4cyclictest27276-21users19:50:262
723699171,8cyclictest0-21swapper/219:40:112
723699171,6cyclictest1127-21sed23:45:262
723699171,6cyclictest0-21swapper/221:30:152
723699171,5cyclictest808-21snmpd21:50:212
7236991712,3cyclictest19397-21acpi00:30:102
723699171,14cyclictest0-21swapper/220:14:072
722599175,5cyclictest18547-21awk23:15:000
722599175,4cyclictest24337-21latency19:45:180
722599171,7cyclictest7-21ksoftirqd/019:40:250
722599171,7cyclictest30714-21/usr/sbin/munin21:15:150
722599171,7cyclictest30714-21/usr/sbin/munin21:15:150
722599171,7cyclictest0-21swapper/000:07:220
722599171,6cyclictest0-21swapper/021:40:220
7225991714,2cyclictest7-21ksoftirqd/023:25:130
7225991713,3cyclictest7-21ksoftirqd/020:35:120
7225991713,3cyclictest0-21swapper/020:31:410
722599171,15cyclictest0-21swapper/023:21:370
722599171,13cyclictest7747-21gltestperf21:35:150
722599170,12cyclictest0-21swapper/000:25:160
143112170,2sleep30-21swapper/319:22:593
724199161,5cyclictest17515-21/usr/sbin/munin20:45:173
724199161,3cyclictest0-21swapper/319:30:113
724199161,12cyclictest808-21snmpd20:30:053
723699169,4cyclictest15934-21/usr/sbin/munin19:25:212
723699164,3cyclictest87650irq/129-enp1s0-20:30:212
723699161,6cyclictest19324-21fschecks_time23:15:142
723699161,5cyclictest0-21swapper/221:05:162
7236991612,3cyclictest0-21swapper/220:16:472
7236991610,3cyclictest827-21runrttasks20:50:232
7236991610,3cyclictest17601-21cpuspeed_turbos20:45:132
722599169,6cyclictest7-21ksoftirqd/023:34:240
722599161,6cyclictest87350irq/126-enp1s0-23:00:190
722599161,6cyclictest8728-21/usr/sbin/munin20:25:180
722599161,6cyclictest0-21swapper/022:30:090
722599161,4cyclictest19712-21/usr/sbin/munin20:50:230
722599161,4cyclictest15316-21/usr/sbin/munin20:40:150
7225991612,3cyclictest0-21swapper/019:15:210
722599161,14cyclictest0-21swapper/021:01:330
722599161,14cyclictest0-21swapper/019:22:550
7225991611,3cyclictest32553-21ls20:05:120
7225991611,3cyclictest32553-21ls20:05:120
7225991611,3cyclictest30414-21expr20:00:130
723699151,5cyclictest13121-21/usr/sbin/munin20:35:122
722599151,6cyclictest0-21swapper/020:45:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional