You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-05 - 00:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sat Jul 04, 2026 12:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
341899309,21cyclictest28-21ksoftirqd/211:14:032
341899261,24cyclictest151rcu_preempt10:36:522
341499260,25cyclictest30768-21diskmemload11:47:261
342599250,24cyclictest781-21ssh10:10:103
342599250,24cyclictest0-21swapper/309:45:273
341899250,24cyclictest30768-21diskmemload09:55:402
341899250,24cyclictest17595-21expr11:51:242
341399250,24cyclictest0-21swapper/010:17:420
342599240,1cyclictest0-21swapper/310:45:553
3418992423,1cyclictest0-21swapper/210:08:542
3418992422,1cyclictest28-21ksoftirqd/209:22:022
3418992422,1cyclictest18234-21ssh10:53:412
3418992422,1cyclictest0-21swapper/209:33:162
3418992419,4cyclictest0-21swapper/211:02:082
341899241,0cyclictest28-21ksoftirqd/210:40:112
341899240,23cyclictest32416-21sed11:06:252
341899240,23cyclictest28-21ksoftirqd/211:16:542
341899240,23cyclictest18022-21munin-run08:45:002
341899240,1cyclictest32476-21cpuspeed11:36:212
341899240,1cyclictest30768-21diskmemload10:22:422
341899240,1cyclictest0-21swapper/212:33:412
3414992423,1cyclictest0-21swapper/111:01:071
341499240,23cyclictest783-21snmpd12:23:481
341499240,23cyclictest30768-21diskmemload09:51:181
341499240,23cyclictest30768-21diskmemload09:37:061
341499240,23cyclictest0-21swapper/111:25:021
341499240,23cyclictest0-21swapper/111:25:021
341499240,23cyclictest0-21swapper/111:14:021
341499240,23cyclictest0-21swapper/109:26:061
341399243,1cyclictest62550irq/127-eno110:03:250
3413992424,0cyclictest0-21swapper/009:22:360
3413992422,1cyclictest4682-21sh11:40:080
3413992422,1cyclictest29419-21sh11:03:560
3413992422,1cyclictest28306-21cron08:00:000
341399242,1cyclictest20093-21munin-plugin-st11:25:000
3413992421,2cyclictest0-21swapper/009:25:230
341399240,24cyclictest0-21swapper/011:25:380
341399240,24cyclictest0-21swapper/011:25:380
341399240,24cyclictest0-21swapper/010:27:080
341399240,23cyclictest0-21swapper/012:25:360
341399240,23cyclictest0-21swapper/012:06:210
341399240,23cyclictest0-21swapper/010:36:330
341399240,1cyclictest2926-21ssh09:13:390
341399240,0cyclictest0-21swapper/012:14:150
3425992323,0cyclictest30768-21diskmemload11:10:463
3425992323,0cyclictest0-21swapper/311:07:303
3425992323,0cyclictest0-21swapper/310:06:483
3425992321,1cyclictest18165-21ssh11:22:193
3425992320,2cyclictest25048-21kernelversion10:31:233
342599230,2cyclictest15433-21turbostat.cron11:50:003
342599230,23cyclictest0-21swapper/312:06:233
342599230,23cyclictest0-21swapper/311:41:393
342599230,23cyclictest0-21swapper/309:56:483
342599230,22cyclictest32130-21sleep12:05:003
342599230,22cyclictest27123-21sh09:35:383
342599230,22cyclictest25122-21ssh11:00:083
342599230,22cyclictest24841-21ssh10:01:283
342599230,22cyclictest20329-21sh09:29:103
342599230,0cyclictest30768-21diskmemload10:52:093
342599230,0cyclictest30768-21diskmemload09:20:353
342599230,0cyclictest151rcu_preempt12:10:393
342599230,0cyclictest0-21swapper/312:31:583
3418992323,0cyclictest0-21swapper/211:56:392
3418992323,0cyclictest0-21swapper/209:47:552
3418992323,0cyclictest0-21swapper/209:25:102
3418992322,1cyclictest6073-21ssh12:10:532
3418992322,1cyclictest2247-21hddtemp_smartct10:11:222
3418992321,1cyclictest0-21swapper/212:36:272
3418992321,1cyclictest0-21swapper/209:20:012
341899231,21cyclictest28-21ksoftirqd/210:47:592
341899230,2cyclictest24149-21rm10:31:122
341899230,23cyclictest6437-21ssh10:15:172
341899230,22cyclictest30768-21diskmemload12:06:312
341899230,22cyclictest30711-21sh11:34:272
341899230,22cyclictest14311-21ssh12:17:422
341899230,22cyclictest0-21swapper/211:28:392
341899230,22cyclictest0-21swapper/211:28:392
341899230,0cyclictest0-21swapper/212:00:042
3414992323,0cyclictest0-21swapper/111:40:151
3414992323,0cyclictest0-21swapper/111:39:391
3414992322,1cyclictest30768-21diskmemload09:41:421
3414992322,1cyclictest17439-21apt-get09:56:201
3414992322,0cyclictest0-21swapper/111:55:031
3414992321,1cyclictest0-21swapper/112:11:211
3414992321,1cyclictest0-21swapper/112:00:441
3414992321,1cyclictest0-21swapper/109:15:541
341499230,23cyclictest0-21swapper/110:13:561
341499230,23cyclictest0-21swapper/109:20:241
341499230,22cyclictest31600-21diskmemload09:10:441
341499230,22cyclictest30838-21ssh10:07:071
341499230,22cyclictest18086-21sh11:22:141
341499230,22cyclictest10424-21sh10:18:271
341499230,22cyclictest0-21swapper/112:35:401
341499230,22cyclictest0-21swapper/110:30:001
341499230,22cyclictest0-21swapper/109:46:201
341499230,1cyclictest0-21swapper/110:22:581
341499230,0cyclictest0-21swapper/110:56:511
341499230,0cyclictest0-21swapper/110:33:481
3413992323,0cyclictest0-21swapper/011:57:160
3413992323,0cyclictest0-21swapper/011:16:020
3413992323,0cyclictest0-21swapper/010:56:110
3413992323,0cyclictest0-21swapper/010:21:330
3413992322,0cyclictest30768-21diskmemload11:11:300
3413992321,1cyclictest26223-21apt-get09:00:020
3413992321,1cyclictest0-21swapper/012:31:110
3413992321,1cyclictest0-21swapper/010:50:590
3413992321,1cyclictest0-21swapper/010:40:130
3413992321,1cyclictest0-21swapper/009:15:590
3413992320,2cyclictest16213-21cron07:35:010
341399230,2cyclictest16444-21ssh11:51:030
341399230,23cyclictest0-21swapper/012:18:200
341399230,23cyclictest0-21swapper/011:30:460
341399230,23cyclictest0-21swapper/010:30:140
341399230,22cyclictest31914-21sendmail09:40:000
341399230,22cyclictest31097-21sh10:07:270
341399230,22cyclictest21830-21sendmail-msp10:00:010
341399230,22cyclictest11903-21cat11:46:240
341399230,22cyclictest10646-21dump-pmu-power09:50:000
341399230,22cyclictest0-21swapper/011:36:010
341399230,1cyclictest3543-21ssh12:36:550
341399230,1cyclictest2421-21irqstats10:11:230
341399230,0cyclictest30768-21diskmemload12:20:070
3425992222,0cyclictest30768-21diskmemload11:30:073
3425992222,0cyclictest0-21swapper/311:35:493
3425992222,0cyclictest0-21swapper/311:27:523
3425992222,0cyclictest0-21swapper/311:27:523
3425992222,0cyclictest0-21swapper/310:35:353
3425992222,0cyclictest0-21swapper/309:11:433
3425992220,1cyclictest6786-21ssh10:15:533
342599220,22cyclictest0-21swapper/312:35:163
342599220,22cyclictest0-21swapper/310:26:003
342599220,22cyclictest0-21swapper/310:23:123
342599220,21cyclictest21784-21diskmemload11:55:483
342599220,21cyclictest20259-21sh10:56:073
342599220,21cyclictest0-21swapper/312:16:293
342599220,21cyclictest0-21swapper/309:50:473
342599220,1cyclictest0-21swapper/309:41:483
342599220,0cyclictest20262-21ssh12:23:083
342599220,0cyclictest0-21swapper/311:19:313
3418992222,0cyclictest0-21swapper/209:12:422
3418992221,1cyclictest0-21swapper/211:20:412
3418992220,1cyclictest32440-21sh09:40:192
3418992220,1cyclictest22511-21ssh10:00:442
3418992220,1cyclictest0-21swapper/208:20:012
3418992219,2cyclictest20726-21ssh10:27:202
3418992218,3cyclictest8586-21awk07:20:002
341899221,1cyclictest28-21ksoftirqd/209:50:582
341899220,2cyclictest20952-21ssh12:24:102
341899220,21cyclictest30768-21diskmemload12:25:182
341899220,21cyclictest26802-21ssh09:35:132
341899220,21cyclictest20416-21munin-plugin-st08:50:002
341899220,21cyclictest19794-21ssh10:55:312
341899220,21cyclictest0-21swapper/211:49:402
341899220,21cyclictest0-21swapper/208:56:202
341899220,21cyclictest0-21swapper/208:01:192
341899220,21cyclictest0-21swapper/207:31:192
3414992222,0cyclictest0-21swapper/112:25:121
3414992222,0cyclictest0-21swapper/110:01:531
3414992220,1cyclictest0-21swapper/111:30:491
3414992220,1cyclictest0-21swapper/110:35:191
3414992220,1cyclictest0-21swapper/109:30:011
341499220,21cyclictest609-21rs:main1
341499220,21cyclictest30768-21diskmemload12:05:051
341499220,21cyclictest30768-21diskmemload10:40:251
341499220,21cyclictest0-21swapper/112:30:121
341499220,21cyclictest0-21swapper/112:15:391
341499220,21cyclictest0-21swapper/111:50:341
341499220,21cyclictest0-21swapper/111:15:401
341499220,21cyclictest0-21swapper/110:50:311
341499220,21cyclictest0-21swapper/110:46:251
341499220,21cyclictest0-21swapper/109:01:201
341499220,21cyclictest0-21swapper/108:56:201
341499220,21cyclictest0-21swapper/108:21:211
341499220,21cyclictest0-21swapper/107:46:201
341499220,21cyclictest0-21swapper/107:20:021
341499220,0cyclictest30768-21diskmemload11:08:501
3413992222,0cyclictest0-21swapper/009:50:460
3413992220,1cyclictest3297-21ssh09:42:230
3413992220,1cyclictest20759-21apt-get08:51:200
341399220,21cyclictest21116-21ssh09:30:010
341399220,21cyclictest19117-21apt-get07:41:190
341399220,1cyclictest0-21swapper/011:07:430
341399220,1cyclictest0-21swapper/010:46:410
3425992121,0cyclictest4144-21apt-get07:11:203
3425992121,0cyclictest0-21swapper/311:50:193
3425992120,1cyclictest22796-21apt-get08:55:023
3425992120,1cyclictest21292-21ssh09:30:143
3425992119,1cyclictest30763-21grep08:05:003
342599210,21cyclictest0-21swapper/312:25:193
342599210,21cyclictest0-21swapper/309:15:163
342599210,20cyclictest6544-21apt-get07:16:203
342599210,20cyclictest18366-21apt-get08:46:203
342599210,1cyclictest18738-21latency_hist07:40:013
342599210,1cyclictest0-21swapper/310:40:533
3418992121,0cyclictest0-21swapper/209:10:012
3418992121,0cyclictest0-21swapper/209:10:012
3418992121,0cyclictest0-21swapper/208:51:202
3418992121,0cyclictest0-21swapper/207:26:202
3418992119,1cyclictest28369-21basename08:00:002
3418992119,1cyclictest0-21swapper/211:40:452
341899210,20cyclictest0-21swapper/209:01:202
341899210,20cyclictest0-21swapper/208:11:182
3414992119,1cyclictest12747-21awk08:35:011
341499210,20cyclictest0-21swapper/108:46:201
342599200,0cyclictest26356-21apt-get09:01:203
342599200,0cyclictest10709-21apt-get08:31:193
3418992020,0cyclictest0-21swapper/208:26:192
3418992018,1cyclictest11407-21basename07:25:002
3418992017,2cyclictest7963-21latency_hist08:25:012
3414992020,0cyclictest0-21swapper/107:16:201
341499200,20cyclictest0-21swapper/107:10:011
341499200,0cyclictest0-21swapper/107:56:201
341499200,0cyclictest0-21swapper/107:41:201
341499200,0cyclictest0-21swapper/107:36:191
3413992020,0cyclictest15133-21apt-get08:40:010
3413992020,0cyclictest0-21swapper/007:51:180
3413992020,0cyclictest0-21swapper/007:21:190
3413992019,1cyclictest15146-21latency_hist08:40:000
3425991919,0cyclictest23902-21apt-get07:51:183
3425991919,0cyclictest0-21swapper/307:31:193
3425991918,1cyclictest10352-21perf08:30:013
3418991919,0cyclictest0-21swapper/207:41:202
341899191,17cyclictest28-21ksoftirqd/207:36:192
3413991919,0cyclictest0-21swapper/008:06:190
3413991919,0cyclictest0-21swapper/008:01:200
3413991919,0cyclictest0-21swapper/007:11:200
3413991917,1cyclictest8575-21apt-get07:20:000
3425991818,0cyclictest0-21swapper/307:56:213
341899181,16cyclictest28-21ksoftirqd/208:36:192
341899180,0cyclictest0-21swapper/207:11:212
3414991818,0cyclictest0-21swapper/108:51:201
3414991818,0cyclictest0-21swapper/107:51:191
3413991818,0cyclictest11753-21apt-get07:26:200
3413991818,0cyclictest0-21swapper/009:06:200
3413991818,0cyclictest0-21swapper/009:06:200
3425991717,0cyclictest5918-21apt-get08:21:213
3425991717,0cyclictest0-21swapper/308:45:003
3425991717,0cyclictest0-21swapper/307:20:023
3418991717,0cyclictest0-21swapper/208:31:192
3418991717,0cyclictest0-21swapper/207:51:182
3414991717,0cyclictest0-21swapper/107:31:191
3413991717,0cyclictest0-21swapper/008:11:190
3413991717,0cyclictest0-21swapper/007:46:200
3425991616,0cyclictest3494-21apt-get08:16:203
342599160,15cyclictest606-21dbus-daemon07:25:073
3418991616,0cyclictest0-21swapper/207:46:202
341899160,15cyclictest31531-21diskstats08:06:212
3414991616,0cyclictest0-21swapper/108:16:201
3414991616,0cyclictest0-21swapper/108:01:201
3414991614,1cyclictest15132-21systemctl08:40:001
3414991613,2cyclictest719-21perf08:10:001
3413991616,0cyclictest0-21swapper/008:56:200
3413991616,0cyclictest0-21swapper/008:26:190
3413991616,0cyclictest0-21swapper/008:16:200
3425991514,1cyclictest31106-21apt-get08:06:193
341399159,3cyclictest313-21systemd-journal08:50:000
3413991514,0cyclictest0-21swapper/008:21:220
3425991414,0cyclictest0-21swapper/309:06:173
3425991414,0cyclictest0-21swapper/309:06:173
3425991414,0cyclictest0-21swapper/308:51:203
342599140,14cyclictest0-21swapper/308:10:573
342599140,14cyclictest0-21swapper/307:46:173
342599140,14cyclictest0-21swapper/307:41:003
342599140,0cyclictest0-21swapper/308:36:283
3414991414,0cyclictest0-21swapper/108:10:391
3414991414,0cyclictest0-21swapper/107:25:011
341499140,0cyclictest0-21swapper/109:06:201
341499140,0cyclictest0-21swapper/109:06:201
3413991414,0cyclictest0-21swapper/007:35:050
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional