You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-25 - 04:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Mon May 25, 2026 00:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
233972236163,23sleep20-21swapper/219:07:202
233962212161,38sleep10-21swapper/119:07:181
233202202168,24sleep00-21swapper/019:06:210
216362201164,25sleep30-21swapper/319:05:033
1883521510,4sleep22370299cyclictest20:10:122
314702630,5sleep22370299cyclictest21:50:252
263202520,2sleep39-21rcu_preempt20:25:233
23695994314,9cyclictest0-21swapper/100:30:141
150892430,1sleep00-21swapper/023:40:230
2369599403,9cyclictest0-21swapper/122:27:311
2370799381,18cyclictest0-21swapper/321:29:263
23702993834,3cyclictest0-21swapper/223:31:232
2370299361,26cyclictest0-21swapper/220:30:042
2370799355,20cyclictest25192-21cstates00:05:163
2370799351,25cyclictest8372-21if_err_enp1s019:45:163
2370799351,25cyclictest8372-21if_err_enp1s019:45:163
2369599331,16cyclictest0-21swapper/123:07:441
133202330,3sleep121-21rcuc/121:10:181
7952320,2sleep00-21swapper/000:22:130
2370799320,18cyclictest0-21swapper/321:30:463
2369599320,15cyclictest0-21swapper/123:40:121
2369099324,13cyclictest13015-21munin-run22:25:010
166092320,2sleep10-21swapper/120:05:131
2370299311,14cyclictest0-21swapper/220:21:082
2370299310,15cyclictest0-21swapper/223:41:082
2369599311,14cyclictest32070-21cat19:25:191
2369099318,11cyclictest13645-21perf23:40:010
2370799301,21cyclictest0-21swapper/321:51:323
2370799301,21cyclictest0-21swapper/321:05:263
23702993027,2cyclictest0-21swapper/220:08:512
23707992912,3cyclictest0-21swapper/323:58:523
2370799291,12cyclictest0-21swapper/321:49:283
2370799291,12cyclictest0-21swapper/300:22:053
2370299291,14cyclictest0-21swapper/221:07:142
2370299290,22cyclictest0-21swapper/223:46:122
2369599291,7cyclictest0-21swapper/120:20:441
2369599291,14cyclictest0-21swapper/123:55:341
2369099292,13cyclictest2650-21munin-run23:15:010
273302280,4sleep121-21rcuc/121:40:251
2370799288,3cyclictest0-21swapper/321:11:263
2370799280,20cyclictest0-21swapper/319:43:083
2370799280,19cyclictest0-21swapper/321:01:303
2370299281,12cyclictest0-21swapper/220:46:402
2370299281,12cyclictest0-21swapper/219:30:582
2370299280,20cyclictest0-21swapper/223:27:002
2369599284,15cyclictest87650irq/129-enp1s0-00:35:161
2369599281,17cyclictest0-21swapper/119:35:061
2369599281,11cyclictest0-21swapper/120:35:161
2370799278,3cyclictest0-21swapper/322:04:523
2370799271,6cyclictest3856-21df_inode00:30:153
23707992711,3cyclictest12390-21diskmemload21:39:223
2370799271,13cyclictest0-21swapper/322:58:163
2370799271,10cyclictest0-21swapper/300:25:153
2370799271,10cyclictest0-21swapper/300:25:153
23707992710,4cyclictest0-21swapper/320:00:143
2370299271,12cyclictest0-21swapper/220:56:102
2370299271,12cyclictest0-21swapper/220:56:102
2369599276,9cyclictest0-21swapper/100:14:021
23695992712,7cyclictest17204-21systemd-journal22:32:311
2369599271,16cyclictest0-21swapper/100:25:121
2369599271,16cyclictest0-21swapper/100:25:121
2369599271,12cyclictest0-21swapper/122:09:221
2369599271,12cyclictest0-21swapper/122:09:221
2369599271,12cyclictest0-21swapper/119:40:201
2370799268,3cyclictest0-21swapper/323:31:203
2370799261,8cyclictest13075-21open_files19:55:223
2370799261,14cyclictest0-21swapper/322:42:483
2370799261,11cyclictest0-21swapper/300:17:573
2370799260,21cyclictest0-21swapper/323:46:323
23702992623,2cyclictest0-21swapper/222:38:102
2370299261,9cyclictest0-21swapper/223:07:302
2370299261,10cyclictest0-21swapper/221:00:102
2370299260,11cyclictest0-21swapper/222:26:122
2369599261,8cyclictest0-21swapper/120:00:301
2369599261,13cyclictest0-21swapper/121:09:521
2369599261,12cyclictest0-21swapper/121:55:581
2369599261,10cyclictest0-21swapper/120:46:121
2370799259,4cyclictest6738-21cpuspeed_turbos22:10:123
2370799259,2cyclictest0-21swapper/322:46:163
2370799256,3cyclictest0-21swapper/322:30:013
2370799255,4cyclictest0-21swapper/322:38:043
2370799251,10cyclictest0-21swapper/323:25:563
2370299258,3cyclictest3029-21taskset00:26:562
2370299258,3cyclictest3029-21taskset00:26:562
2370299251,12cyclictest0-21swapper/219:56:392
2370299251,11cyclictest0-21swapper/219:44:272
2370299251,10cyclictest0-21swapper/220:43:322
2370299251,10cyclictest0-21swapper/219:53:452
2370299251,10cyclictest0-21swapper/219:47:032
2370299251,10cyclictest0-21swapper/219:47:032
2369599259,4cyclictest2079-21timerwakeupswit23:10:241
2369599258,9cyclictest13980-21perf20:00:011
2369599256,9cyclictest27022-21gltestperf22:55:141
2369599255,11cyclictest16235-21sed21:15:261
2369599251,2cyclictest87550irq/128-enp1s0-00:05:401
2369599251,13cyclictest0-21swapper/120:25:561
2369599251,12cyclictest0-21swapper/120:33:241
2369599251,11cyclictest0-21swapper/120:57:041
2369599251,11cyclictest0-21swapper/120:57:041
2369599251,10cyclictest24519-21gltestperf19:10:161
2369099250,21cyclictest8843-21/usr/sbin/munin22:15:200
2370799249,4cyclictest0-21swapper/323:14:123
2370799249,3cyclictest0-21swapper/323:17:243
2370799249,3cyclictest0-21swapper/323:17:243
2370799248,8cyclictest0-21swapper/320:58:023
2370799248,8cyclictest0-21swapper/320:58:023
2370799246,3cyclictest0-21swapper/321:23:463
2370799244,3cyclictest0-21swapper/321:55:243
23707992418,3cyclictest0-21swapper/320:19:593
2370799241,13cyclictest0-21swapper/323:22:203
2370299241,12cyclictest0-21swapper/222:16:082
2370299241,12cyclictest0-21swapper/220:53:482
2370299241,10cyclictest0-21swapper/223:51:232
23702992410,3cyclictest0-21swapper/220:27:102
2370299240,13cyclictest0-21swapper/222:30:382
2369599245,3cyclictest0-21swapper/121:36:431
23695992414,3cyclictest0-21swapper/122:15:241
2369599241,16cyclictest0-21swapper/123:47:081
2369099247,7cyclictest12404-21idleruntime-cro21:10:020
2369099243,7cyclictest14829-21users22:25:260
2369099241,14cyclictest755-21rngd00:15:240
2369099241,11cyclictest17645-21users23:45:260
2370799239,3cyclictest0-21swapper/319:16:143
2370799238,3cyclictest0-21swapper/300:01:023
2370799237,3cyclictest0-21swapper/323:43:203
23707992318,3cyclictest0-21swapper/320:40:343
2370799231,13cyclictest28412-21switchtime00:10:253
23707992310,3cyclictest0-21swapper/320:34:033
2370299237,3cyclictest0-21swapper/222:23:202
2370299231,9cyclictest0-21swapper/222:56:282
2370299231,8cyclictest0-21swapper/219:38:562
2370299231,12cyclictest0-21swapper/200:39:162
2370299231,11cyclictest24231-21df_inode19:10:142
2370299231,11cyclictest0-21swapper/221:31:302
2370299231,10cyclictest0-21swapper/200:12:362
2369599232,3cyclictest7043-21munin-run23:25:021
2369599232,12cyclictest32717-21/usr/sbin/munin00:20:241
23695992317,3cyclictest3216-21df_inode23:15:141
23695992317,3cyclictest3216-21df_inode23:15:141
2369599231,13cyclictest0-21swapper/122:23:441
2369599231,12cyclictest0-21swapper/121:54:441
2369599231,11cyclictest0-21swapper/123:25:151
2369099234,8cyclictest0-21swapper/021:15:020
2370799228,3cyclictest0-21swapper/323:35:483
2370799228,3cyclictest0-21swapper/320:54:183
2370799227,3cyclictest0-21swapper/322:51:323
2370799226,3cyclictest0-21swapper/321:17:103
2370799225,3cyclictest0-21swapper/320:20:513
2370799225,3cyclictest0-21swapper/319:30:343
2370799223,3cyclictest0-21swapper/323:07:083
2370799222,3cyclictest0-21swapper/320:37:113
2370799221,9cyclictest0-21swapper/320:07:363
2370799221,6cyclictest0-21swapper/323:54:283
23707992210,3cyclictest0-21swapper/319:54:463
2370299229,3cyclictest0-21swapper/221:18:422
2370299228,3cyclictest0-21swapper/222:07:132
2370299228,3cyclictest0-21swapper/222:07:132
2370299227,2cyclictest0-21swapper/219:21:462
2370299225,2cyclictest0-21swapper/222:45:002
2370299221,9cyclictest0-21swapper/223:24:242
2370299221,11cyclictest0-21swapper/222:11:082
2370299221,10cyclictest0-21swapper/220:39:262
2370299221,10cyclictest0-21swapper/220:18:302
2370299220,8cyclictest0-21swapper/221:25:162
2369599226,9cyclictest0-21swapper/120:11:221
2369599221,3cyclictest27224-21fschecks_count19:15:141
2369599221,16cyclictest0-21swapper/123:38:391
2369599221,15cyclictest0-21swapper/123:52:111
2369599221,10cyclictest0-21swapper/123:00:591
2369599221,10cyclictest0-21swapper/122:01:171
2369099226,8cyclictest0-21swapper/019:55:170
2369099224,8cyclictest26995-21sort00:10:010
2369099223,10cyclictest20451-21/usr/sbin/munin23:55:160
2369099222,11cyclictest81ktimersoftd/000:10:150
2369099221,5cyclictest0-21swapper/022:14:390
238022210,6sleep22370299cyclictest00:00:242
2370799218,3cyclictest12390-21diskmemload22:22:293
2370799218,3cyclictest0-21swapper/320:47:223
2370799217,3cyclictest0-21swapper/321:40:543
2370799217,3cyclictest0-21swapper/320:13:513
2370799214,10cyclictest0-21swapper/319:13:193
2370799212,10cyclictest0-21swapper/319:36:583
2370799211,13cyclictest0-21swapper/323:00:043
2370799211,11cyclictest6201-21tune2fs00:35:173
2370799211,11cyclictest0-21swapper/322:15:273
2370299215,3cyclictest0-21swapper/223:57:332
2370299214,3cyclictest0-21swapper/200:20:522
2370299211,9cyclictest0-21swapper/219:17:342
2370299210,10cyclictest0-21swapper/223:16:422
2370299210,10cyclictest0-21swapper/223:16:422
2370299210,10cyclictest0-21swapper/200:34:042
2369599218,3cyclictest21737-21sshd22:44:171
2369599214,11cyclictest87350irq/126-enp1s0-20:54:461
2369599211,9cyclictest0-21swapper/100:18:131
2369599211,18cyclictest19061-21timerwakeupswit22:35:251
2369599211,14cyclictest0-21swapper/121:45:571
2369599211,13cyclictest7111-21tune2fs22:10:151
2369599211,12cyclictest0-21swapper/120:40:161
2369599211,11cyclictest23573-21memory00:00:201
2369599211,11cyclictest0-21swapper/121:29:411
2369599211,10cyclictest0-21swapper/122:50:501
23695992110,3cyclictest22057-21smart_sda20:15:251
2369099217,3cyclictest30935-21kworker/0:200:30:220
2369099216,9cyclictest87550irq/128-enp1s0-23:30:130
2369099216,14cyclictest0-21swapper/023:25:010
2369099215,4cyclictest808-21snmpd20:35:260
2369099211,6cyclictest0-21swapper/021:18:560
2370799207,3cyclictest0-21swapper/319:25:423
2370799205,3cyclictest0-21swapper/319:20:103
2370799203,3cyclictest5916-21unixbench-2d22:05:263
2370799203,3cyclictest5916-21unixbench-2d22:05:263
2370299206,3cyclictest0-21swapper/223:00:122
2370299206,3cyclictest0-21swapper/222:53:562
2370299205,3cyclictest0-21swapper/200:18:442
2370299204,3cyclictest0-21swapper/219:28:022
2370299201,9cyclictest0-21swapper/223:37:002
2370299201,12cyclictest0-21swapper/221:42:262
2370299200,13cyclictest0-21swapper/221:57:192
2370299200,10cyclictest0-21swapper/220:01:302
2370299200,10cyclictest0-21swapper/200:06:522
23695992015,3cyclictest10875-21ntp_states19:50:221
23695992012,2cyclictest0-21swapper/123:30:171
2369599201,17cyclictest0-21swapper/119:20:421
2369599201,14cyclictest0-21swapper/121:33:161
2369599201,12cyclictest0-21swapper/119:31:001
2369599201,11cyclictest0-21swapper/122:48:091
2369099209,6cyclictest1128-40dnf20:42:040
2369099203,2cyclictest26188-21cut20:25:190
2369099202,5cyclictest9804-21/usr/sbin/munin19:50:240
2369099201,7cyclictest27023-21grep22:55:150
2369099201,5cyclictest0-21swapper/022:36:190
2369099201,12cyclictest0-21swapper/000:28:390
2369099201,12cyclictest0-21swapper/000:28:390
2369099201,11cyclictest30834-21/usr/sbin/munin23:05:220
2369099201,11cyclictest18597-21/usr/sbin/munin20:10:170
2369099201,11cyclictest0-21swapper/021:45:400
2370299198,3cyclictest0-21swapper/221:47:242
2370299197,3cyclictest0-21swapper/221:10:182
2370299193,7cyclictest12390-21diskmemload22:00:152
2369599191,3cyclictest0-21swapper/121:04:521
2369599191,2cyclictest0-21swapper/121:23:071
2369599191,11cyclictest7604-21/usr/sbin/munin19:45:191
2369599191,11cyclictest7604-21/usr/sbin/munin19:45:191
2369099197,3cyclictest2872-21/usr/sbin/munin23:15:160
2369099197,3cyclictest2872-21/usr/sbin/munin23:15:160
2369099195,3cyclictest3612-21perf20:50:010
2369099194,13cyclictest23101-21ntpq22:45:230
2369099192,5cyclictest0-21swapper/021:25:120
2369099191,9cyclictest0-21swapper/023:00:250
2369099191,4cyclictest0-21swapper/022:01:290
2369099191,11cyclictest0-21swapper/021:40:170
23690991910,6cyclictest8583-21fschecks_count21:00:160
23690991910,2cyclictest87550irq/128-enp1s0-20:05:210
2370799181,8cyclictest0-21swapper/322:31:203
2370299185,3cyclictest0-21swapper/221:23:102
2370299180,9cyclictest0-21swapper/222:46:562
2369099185,2cyclictest22536-21pmu-power21:30:220
2369099185,11cyclictest1-21systemd19:19:170
2369099184,8cyclictest87550irq/128-enp1s0-21:55:160
2369099183,8cyclictest0-21swapper/022:30:490
2369099183,4cyclictest6027-21cstates00:35:140
2369099183,2cyclictest31187-21cpu19:25:120
2369099182,2cyclictest12141-21kworker/0:221:20:240
2369099181,9cyclictest3554-21sh19:35:140
2369099181,8cyclictest6276-21sshd22:08:170
2369099181,8cyclictest6276-21sshd22:08:170
2369099181,7cyclictest0-21swapper/022:40:320
2369099181,3cyclictest7478-21seq20:55:260
2369099181,3cyclictest7478-21seq20:55:260
23690991813,3cyclictest22771-21perf20:20:010
2369099181,2cyclictest87550irq/128-enp1s0-19:45:160
2369099181,2cyclictest87550irq/128-enp1s0-19:45:160
2369099180,2cyclictest87550irq/128-enp1s0-23:26:370
2370299176,3cyclictest0-21swapper/221:35:462
2369099174,4cyclictest808-21snmpd20:30:250
2369099174,4cyclictest1546-21tune2fs19:30:150
2369099173,7cyclictest0-21swapper/000:03:430
2369099171,7cyclictest28973-21/usr/sbin/munin19:20:120
2369099171,6cyclictest0-21swapper/023:50:170
2369099171,6cyclictest0-21swapper/021:51:210
2369099171,6cyclictest0-21swapper/021:37:270
2369099171,14cyclictest7396-21perf19:45:010
2369099170,8cyclictest0-21swapper/022:50:170
23702991611,3cyclictest986-21df23:10:122
23690991611,3cyclictest4564-21if_enp2s020:50:180
23690991611,3cyclictest23574-21grep20:20:150
2369099158,4cyclictest14809-21grep20:00:140
2369099156,7cyclictest25956-21sed19:15:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional