You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-28 - 15:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Tue Jul 28, 2026 12:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20072236161,25sleep00-21swapper/007:05:020
40402233163,57sleep20-21swapper/207:08:222
39682206178,19sleep30-21swapper/307:07:273
38482202165,24sleep10-21swapper/107:05:521
66332570,1sleep10-21swapper/112:05:241
314102550,4sleep230-21rcuc/209:20:252
425899388,14cyclictest7535-21munin-run12:10:000
425899368,13cyclictest5345-21awk12:05:010
191922360,1sleep20-21swapper/208:55:132
191922360,1sleep20-21swapper/208:55:132
96352330,3sleep1426299cyclictest11:00:131
4262993318,8cyclictest0-21swapper/111:35:381
427299321,11cyclictest0-21swapper/309:22:593
427299320,23cyclictest0-21swapper/307:15:033
152102320,9sleep123-21ksoftirqd/110:00:021
16682310,1sleep10-21swapper/108:15:151
427299300,12cyclictest0-21swapper/307:55:193
426799301,7cyclictest0-21swapper/212:13:072
4258993012,15cyclictest3341-21awk08:20:010
427299291,9cyclictest25321-21diskmemload11:49:273
427299291,13cyclictest0-21swapper/308:25:283
427299291,12cyclictest25321-21diskmemload11:15:313
427299291,12cyclictest0-21swapper/312:11:553
427299290,11cyclictest0-21swapper/310:45:163
4262992919,3cyclictest0-21swapper/110:08:161
4262992919,3cyclictest0-21swapper/110:08:161
427299281,12cyclictest0-21swapper/309:15:313
427299281,11cyclictest0-21swapper/307:45:073
427299281,10cyclictest0-21swapper/309:02:273
427299280,13cyclictest0-21swapper/311:23:313
427299280,13cyclictest0-21swapper/311:23:313
427299280,13cyclictest0-21swapper/310:58:313
427299280,13cyclictest0-21swapper/310:14:193
427299280,11cyclictest0-21swapper/311:05:233
427299280,10cyclictest0-21swapper/312:06:313
427299280,10cyclictest0-21swapper/309:45:113
426799281,5cyclictest0-21swapper/210:34:012
4258992810,15cyclictest11517-21munin-run07:25:020
149032280,4sleep0425899cyclictest07:30:190
427299271,11cyclictest25321-21diskmemload12:01:433
427299271,11cyclictest0-21swapper/311:50:033
427299271,11cyclictest0-21swapper/307:11:273
427299270,12cyclictest0-21swapper/310:27:313
427299270,11cyclictest0-21swapper/311:25:593
427299270,11cyclictest0-21swapper/309:56:113
426799271,5cyclictest28010-21kworker/u8:010:48:562
426299271,12cyclictest12180-21hddtemp_smartct11:05:171
426299270,5cyclictest0-21swapper/111:13:061
426299270,5cyclictest0-21swapper/109:13:531
426299270,4cyclictest0-21swapper/110:48:281
426299270,4cyclictest0-21swapper/110:00:031
427299261,9cyclictest0-21swapper/309:38:593
427299261,11cyclictest7965-21grep09:40:253
427299261,10cyclictest0-21swapper/311:40:513
427299261,10cyclictest0-21swapper/310:22:393
426799267,9cyclictest0-21swapper/211:45:142
426799265,11cyclictest0-21swapper/212:00:162
4262992613,3cyclictest0-21swapper/110:55:251
426299261,11cyclictest30910-21users08:05:261
426299260,5cyclictest0-21swapper/109:27:491
426299260,5cyclictest0-21swapper/109:20:001
426299260,4cyclictest0-21swapper/110:16:041
427299258,3cyclictest0-21swapper/308:11:473
427299255,8cyclictest0-21swapper/308:44:353
427299251,8cyclictest0-21swapper/311:57:153
427299251,10cyclictest0-21swapper/307:37:153
427299250,9cyclictest0-21swapper/311:32:473
427299250,12cyclictest0-21swapper/310:39:523
427299250,11cyclictest0-21swapper/312:29:473
427299250,11cyclictest0-21swapper/312:22:273
427299250,11cyclictest0-21swapper/309:33:313
427299250,11cyclictest0-21swapper/308:31:363
427299250,10cyclictest0-21swapper/312:39:353
427299250,10cyclictest0-21swapper/312:39:353
427299250,10cyclictest0-21swapper/312:15:233
427299250,10cyclictest0-21swapper/310:30:193
427299250,10cyclictest0-21swapper/309:26:313
427299250,10cyclictest0-21swapper/307:53:473
426799252,6cyclictest28835-21ntp_states09:15:212
4267992512,8cyclictest5059-21unixbench_multi08:20:252
426299251,8cyclictest0-21swapper/111:51:301
426299251,7cyclictest0-21swapper/110:54:591
426299251,7cyclictest0-21swapper/109:42:361
426299251,17cyclictest0-21swapper/112:01:001
426299251,13cyclictest0-21swapper/111:32:291
426299251,11cyclictest0-21swapper/109:52:281
426299250,9cyclictest0-21swapper/109:45:231
426299250,9cyclictest0-21swapper/109:34:361
426299250,8cyclictest0-21swapper/112:20:351
426299250,8cyclictest0-21swapper/111:17:311
426299250,7cyclictest0-21swapper/110:41:221
426299250,5cyclictest0-21swapper/112:32:101
426299250,5cyclictest0-21swapper/111:48:121
426299250,5cyclictest0-21swapper/110:14:531
427299241,9cyclictest25321-21diskmemload10:43:203
427299241,9cyclictest0-21swapper/310:17:013
427299241,8cyclictest0-21swapper/312:34:513
427299241,7cyclictest0-21swapper/307:33:533
427299240,9cyclictest0-21swapper/310:01:593
427299240,8cyclictest0-21swapper/309:51:393
427299240,11cyclictest0-21swapper/308:15:113
427299240,10cyclictest0-21swapper/308:51:353
426799241,7cyclictest17385-21irqstats12:30:182
426799241,6cyclictest0-21swapper/210:08:162
426799241,6cyclictest0-21swapper/210:08:162
426299241,11cyclictest19158-21ntp_states11:20:201
426299241,11cyclictest19158-21ntp_states11:20:201
426299241,11cyclictest0-21swapper/110:20:481
4262992410,9cyclictest0-21swapper/112:11:221
426299240,8cyclictest0-21swapper/110:31:081
426299240,8cyclictest0-21swapper/109:37:281
426299240,6cyclictest0-21swapper/111:58:321
426299240,6cyclictest0-21swapper/110:28:421
426299240,5cyclictest0-21swapper/110:38:021
426299240,4cyclictest0-21swapper/112:27:221
426299240,4cyclictest0-21swapper/112:17:311
426299240,3cyclictest0-21swapper/111:40:331
426299240,11cyclictest0-21swapper/109:23:491
427299237,3cyclictest0-21swapper/307:29:193
427299236,3cyclictest0-21swapper/307:20:333
427299236,10cyclictest0-21swapper/308:08:273
427299231,9cyclictest0-21swapper/308:20:233
427299230,9cyclictest0-21swapper/311:37:073
427299230,8cyclictest0-21swapper/308:56:113
427299230,8cyclictest0-21swapper/308:56:113
427299230,10cyclictest0-21swapper/309:14:273
426799231,4cyclictest0-21swapper/210:20:452
426799230,4cyclictest0-21swapper/211:35:382
426299231,14cyclictest0-21swapper/111:29:541
426299231,11cyclictest11470-21switchtime08:35:241
426299230,5cyclictest0-21swapper/112:37:441
426299230,5cyclictest0-21swapper/112:37:441
426299230,14cyclictest0-21swapper/107:10:071
425899236,15cyclictest9944-21awk08:35:000
427299226,6cyclictest0-21swapper/310:53:073
427299225,3cyclictest0-21swapper/307:44:413
427299225,3cyclictest0-21swapper/307:44:413
427299221,12cyclictest14554-21/usr/sbin/munin08:45:173
427299221,11cyclictest0-21swapper/308:37:243
427299220,10cyclictest0-21swapper/310:07:293
427299220,10cyclictest0-21swapper/310:07:293
426299221,7cyclictest0-21swapper/108:20:251
4262992215,4cyclictest17517-21timerandwakeup07:35:271
426299221,3cyclictest13419-21ntp_states08:40:221
426299221,18cyclictest0-21swapper/108:00:151
427299211,9cyclictest0-21swapper/311:03:273
427299210,9cyclictest0-21swapper/311:10:353
426799211,6cyclictest18245-1kworker/u9:211:21:462
426799211,6cyclictest18245-1kworker/u9:211:21:462
425899219,3cyclictest4454-21/usr/sbin/munin07:10:160
425899216,3cyclictest87550irq/128-enp1s0-10:00:010
4258992114,5cyclictest0-21swapper/008:00:230
425899211,11cyclictest9953-21/usr/sbin/munin12:15:160
425899210,4cyclictest0-21swapper/009:18:490
426799209,2cyclictest9086-21awk11:00:012
426799206,5cyclictest0-21swapper/212:25:202
426799205,6cyclictest5542-21gltestperf10:50:162
426799205,4cyclictest14004-21cpuspeed_turbos07:30:122
426799204,4cyclictest5763-21/usr/sbin/munin08:25:202
426799201,5cyclictest0-21swapper/211:13:352
4267992014,3cyclictest6564-21chrt09:40:122
426799201,11cyclictest0-21swapper/208:30:252
426299209,8cyclictest19905-21unixbench_singl07:40:261
426299209,8cyclictest19905-21unixbench_singl07:40:261
426299201,6cyclictest0-21swapper/109:00:281
426299201,4cyclictest0-21swapper/107:45:121
426299201,3cyclictest0-21swapper/107:50:191
426299201,2cyclictest0-21swapper/109:07:151
426299201,17cyclictest0-21swapper/107:55:091
425899202,10cyclictest12483-21df12:20:130
425899201,6cyclictest0-21swapper/009:34:180
425899201,4cyclictest0-21swapper/012:25:230
425899201,3cyclictest6877-21munin-run10:55:000
425899201,13cyclictest0-21swapper/009:44:070
425899201,11cyclictest4639-21sshd10:49:430
425899201,11cyclictest0-21swapper/011:26:430
425899201,11cyclictest0-21swapper/010:44:110
425899201,11cyclictest0-21swapper/010:16:270
427299194,3cyclictest0-21swapper/309:09:133
427299190,9cyclictest0-21swapper/308:00:313
426799194,4cyclictest32634-21munin-run10:40:012
426799191,8cyclictest12222-21basename11:05:182
426799191,7cyclictest0-21swapper/211:57:262
426799191,5cyclictest32229-21missed_timers11:50:192
426799191,4cyclictest763-21dbus-daemon12:38:522
426799191,4cyclictest763-21dbus-daemon12:38:522
426799191,15cyclictest29101-21dump-pmu-power08:05:012
426799191,15cyclictest21153-21grep10:10:262
426799191,10cyclictest0-21swapper/212:19:522
426799190,4cyclictest21977-21kworker/u8:212:20:142
426299194,12cyclictest8209-21cstates08:30:131
426299191,6cyclictest10691-21pmu-power07:20:221
4262991915,2cyclictest15608-21ntpq08:45:211
426299191,17cyclictest0-21swapper/107:32:411
426299191,16cyclictest0-21swapper/108:55:161
426299191,16cyclictest0-21swapper/108:55:161
426299191,16cyclictest0-21swapper/107:25:111
426299190,17cyclictest0-21swapper/107:15:101
425899198,4cyclictest25031-21needreboot10:20:190
425899195,6cyclictest18245-1kworker/u9:209:23:410
425899193,3cyclictest87550irq/128-enp1s0-11:10:170
425899191,7cyclictest32120-21gltestperf08:10:160
425899191,6cyclictest17740-21strings08:50:190
425899191,6cyclictest0-21swapper/011:55:590
425899191,6cyclictest0-21swapper/009:36:300
425899191,5cyclictest0-21swapper/009:54:590
425899191,4cyclictest9301-21smartctl09:45:160
425899191,16cyclictest0-21swapper/008:05:120
425899191,15cyclictest13015-21sed11:05:270
425899191,12cyclictest0-21swapper/012:31:030
425899191,11cyclictest27437-21sshd09:13:510
425899191,10cyclictest0-21swapper/009:27:260
425899190,10cyclictest0-21swapper/010:01:280
60292181,3sleep20-21swapper/209:38:042
5182181,2sleep00-21swapper/011:51:180
426799185,3cyclictest12050irq/123-ahci[0010:40:212
426799182,3cyclictest0-21swapper/207:27:062
426799182,15cyclictest0-21swapper/210:17:472
426799182,15cyclictest0-21swapper/208:38:492
426799181,7cyclictest28174-21smart_sda11:40:242
426799181,7cyclictest17992-21cut11:20:012
426799181,7cyclictest0-21swapper/209:47:092
426799181,6cyclictest0-21swapper/209:50:022
426799181,5cyclictest6736-21sed07:15:022
426799181,5cyclictest30249-21expr08:05:202
426799181,5cyclictest26829-21switchtime09:10:252
426799181,5cyclictest24926-21/usr/sbin/munin07:55:182
426799181,4cyclictest24692-21latency_hist07:55:012
426799181,4cyclictest0-21swapper/209:55:082
426799181,3cyclictest0-21swapper/210:28:152
4267991813,3cyclictest14554-21/usr/sbin/munin08:45:252
4267991812,3cyclictest3087-21sh09:30:212
4267991812,3cyclictest15772-21sort10:00:152
426799181,10cyclictest0-21swapper/211:34:212
426299181,9cyclictest17564-21/usr/sbin/munin08:50:171
4262991814,3cyclictest0-21swapper/108:13:211
426299180,17cyclictest0-21swapper/108:26:141
425899188,3cyclictest12353-21/usr/sbin/munin08:40:130
425899184,7cyclictest9612-21taskset12:13:430
425899183,8cyclictest9041-21kworker/u8:111:15:030
425899183,12cyclictest28436-21/usr/sbin/munin10:30:160
425899181,9cyclictest808-21snmpd07:37:330
425899181,9cyclictest23583-21ntp_states11:30:210
425899181,8cyclictest17787-21df10:05:150
425899181,8cyclictest17787-21df10:05:150
425899181,4cyclictest19843-21ntp_states12:35:230
425899181,4cyclictest19843-21ntp_states12:35:230
4258991811,4cyclictest20134-21fschecks_time10:10:160
425899181,10cyclictest0-21swapper/011:04:530
426799172,7cyclictest30-21rcuc/208:40:192
426799171,8cyclictest0-21swapper/208:10:222
426799171,7cyclictest17158-21fschecks_count08:50:142
426799171,5cyclictest32183-21/usr/sbin/munin09:25:252
426799171,5cyclictest0-21swapper/212:05:422
426799171,5cyclictest0-21swapper/211:25:162
4267991712,3cyclictest10368-21ntp_states11:00:212
426799171,15cyclictest0-21swapper/207:38:112
425899177,3cyclictest9285-21sshd07:19:470
425899175,3cyclictest7315-21cstates10:55:120
425899175,3cyclictest30819-21sshd11:46:510
425899172,5cyclictest25504-21perl07:55:160
425899172,3cyclictest0-21swapper/010:35:180
425899171,9cyclictest24816-21cut11:35:140
425899171,9cyclictest0-21swapper/007:43:340
425899171,9cyclictest0-21swapper/007:43:340
425899171,8cyclictest808-21snmpd07:52:440
425899171,8cyclictest0-21swapper/011:42:320
425899171,6cyclictest25319-21crond09:10:010
425899171,5cyclictest0-21swapper/010:29:020
425899171,4cyclictest0-21swapper/008:24:390
4258991712,3cyclictest14618-21cpuspeed_turbos08:45:120
425899171,15cyclictest0-21swapper/011:22:300
425899171,15cyclictest0-21swapper/011:22:300
425899171,13cyclictest21147-21/usr/sbin/munin09:00:130
425899171,13cyclictest11743-21/usr/sbin/munin07:25:140
426799165,3cyclictest87550irq/128-enp1s0-07:45:142
426799162,7cyclictest0-21swapper/207:14:262
426799161,7cyclictest0-21swapper/209:05:242
426799161,6cyclictest19596-21smart_sda07:40:232
426799161,6cyclictest19596-21smart_sda07:40:232
426799161,6cyclictest0-21swapper/208:15:172
426799161,5cyclictest0-21swapper/209:00:232
426799161,14cyclictest0-21swapper/207:21:462
425899161,5cyclictest5763-21/usr/sbin/munin08:25:120
425899161,2cyclictest0-21swapper/007:47:090
4258991612,3cyclictest0-21swapper/008:59:280
4258991612,3cyclictest0-21swapper/008:59:280
425899151,8cyclictest87550irq/128-enp1s0-08:35:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional