You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-25 - 10:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Tue Aug 25, 2026 00:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
18862236162,25sleep00-21swapper/019:07:560
20452203167,23sleep20-21swapper/219:09:582
18082203166,24sleep10-21swapper/119:06:551
17132202165,24sleep30-21swapper/319:05:413
2823021840,5sleep1213699cyclictest21:20:161
246522670,1sleep10-21swapper/123:40:021
2141995148,2cyclictest0-21swapper/223:58:242
45982500,2sleep0212899cyclictest22:54:340
213699341,30cyclictest0-21swapper/122:55:221
2136993214,10cyclictest32561-21diskstats21:30:151
126562320,7sleep2311ktimersoftd/220:45:132
125452320,5sleep3214899cyclictest23:10:213
213699312,17cyclictest13522-21kworker/u8:121:03:411
213699312,17cyclictest13522-21kworker/u8:121:03:411
212899316,12cyclictest1827-21munin-run21:35:010
214199301,6cyclictest0-21swapper/222:55:222
2136993012,10cyclictest16206-21df23:20:131
2136993012,10cyclictest16206-21df23:20:131
212899302,12cyclictest6234-21awk21:45:020
213699291,22cyclictest0-21swapper/121:26:101
214199281,5cyclictest0-21swapper/222:03:012
213699281,7cyclictest13522-21kworker/u8:122:15:151
214199271,18cyclictest0-21swapper/223:35:332
2136992723,3cyclictest0-21swapper/121:05:161
2136992721,3cyclictest26886-21smart_sda21:15:271
214199269,8cyclictest1200-21awk20:20:012
214199263,4cyclictest0-21swapper/222:17:572
214199261,6cyclictest4602-21diskstats21:40:152
2136992620,3cyclictest15883-21/usr/sbin/munin20:50:241
213699260,21cyclictest0-21swapper/122:03:251
214199251,5cyclictest0-21swapper/200:01:232
213699251,22cyclictest0-21swapper/121:10:141
213699251,18cyclictest0-21swapper/120:44:101
213699250,3cyclictest0-21swapper/100:38:421
213699250,3cyclictest0-21swapper/100:38:421
213699250,2cyclictest6073-21kworker/u8:221:55:001
213699250,18cyclictest0-21swapper/121:48:591
212899251,13cyclictest32603-21munin-run22:45:020
214899241,12cyclictest0-21swapper/300:33:513
214199246,9cyclictest87650irq/129-enp1s0-22:05:232
214199246,4cyclictest2448-21munin-run22:50:022
214199246,4cyclictest0-21swapper/223:53:482
214199242,4cyclictest0-21swapper/200:05:242
214199241,6cyclictest0-21swapper/200:24:052
214199241,5cyclictest0-21swapper/223:07:532
214199241,17cyclictest0-21swapper/223:23:512
214199241,17cyclictest0-21swapper/223:23:512
214199240,4cyclictest0-21swapper/221:55:012
213699248,3cyclictest15503-21cstates22:05:141
2136992421,2cyclictest0-21swapper/123:27:391
2136992421,2cyclictest0-21swapper/120:16:071
2136992421,2cyclictest0-21swapper/119:48:561
2136992420,3cyclictest0-21swapper/123:10:371
213699241,9cyclictest0-21swapper/122:30:051
213699241,6cyclictest0-21swapper/123:55:491
213699240,9cyclictest0-21swapper/122:36:001
213699240,5cyclictest0-21swapper/121:37:541
213699240,2cyclictest0-21swapper/100:05:391
213699240,22cyclictest0-21swapper/123:00:201
213699240,22cyclictest0-21swapper/119:25:021
213699240,10cyclictest0-21swapper/122:47:231
212899244,2cyclictest16498-21kworker/0:320:58:570
2128992420,3cyclictest0-21swapper/022:30:190
214899238,4cyclictest87350irq/126-enp1s0-22:05:223
214899232,4cyclictest0-21swapper/322:50:143
214899232,3cyclictest41-21ksoftirqd/321:30:173
214899231,7cyclictest0-21swapper/322:43:323
214899231,6cyclictest0-21swapper/300:11:053
214899231,10cyclictest0-21swapper/323:21:193
214899231,10cyclictest0-21swapper/323:21:193
214899230,9cyclictest0-21swapper/319:10:513
214199239,3cyclictest3820-21cstates20:25:142
214199235,9cyclictest23203-21diskmemload22:10:532
214199235,9cyclictest23203-21diskmemload22:10:532
214199232,6cyclictest13522-21kworker/u8:121:37:532
214199231,5cyclictest0-21swapper/223:31:312
214199231,5cyclictest0-21swapper/222:33:252
214199231,5cyclictest0-21swapper/222:22:332
213699236,7cyclictest0-21swapper/100:25:241
213699231,20cyclictest0-21swapper/100:15:481
213699231,19cyclictest26011-21sa122:30:011
213699231,10cyclictest0-21swapper/123:52:401
213699231,10cyclictest0-21swapper/100:10:031
213699230,8cyclictest0-21swapper/123:05:191
213699230,7cyclictest0-21swapper/100:21:341
213699230,6cyclictest0-21swapper/122:53:081
213699230,5cyclictest0-21swapper/122:12:481
213699230,5cyclictest0-21swapper/122:12:481
213699230,3cyclictest0-21swapper/123:31:311
213699230,3cyclictest0-21swapper/100:01:221
213699230,21cyclictest0-21swapper/120:05:021
213699230,10cyclictest0-21swapper/122:43:291
2128992319,3cyclictest0-21swapper/021:20:190
214899227,4cyclictest0-21swapper/322:02:433
214899221,7cyclictest0-21swapper/323:58:033
214899221,6cyclictest0-21swapper/300:02:023
214899221,5cyclictest0-21swapper/321:51:403
214899220,9cyclictest0-21swapper/323:06:163
214899220,9cyclictest0-21swapper/321:42:443
214199228,2cyclictest87550irq/128-enp1s0-21:34:052
214199224,8cyclictest87350irq/126-enp1s0-19:50:222
214199224,8cyclictest87350irq/126-enp1s0-19:50:212
214199223,4cyclictest0-21swapper/222:28:452
214199223,4cyclictest0-21swapper/221:16:562
214199221,7cyclictest0-21swapper/221:47:552
214199221,7cyclictest0-21swapper/221:21:352
214199221,6cyclictest0-21swapper/223:18:132
214199221,6cyclictest0-21swapper/223:00:522
214199221,6cyclictest0-21swapper/222:53:432
214199221,6cyclictest0-21swapper/222:40:142
214199221,5cyclictest0-21swapper/223:26:462
214199221,5cyclictest0-21swapper/200:35:532
214199221,5cyclictest0-21swapper/200:35:532
214199221,5cyclictest0-21swapper/200:15:452
214199221,13cyclictest0-21swapper/223:41:212
214199221,11cyclictest8030-21/usr/sbin/munin20:35:242
213699224,3cyclictest0-21swapper/123:15:131
213699223,16cyclictest755-21rngd23:46:081
2136992212,3cyclictest25963-21smart_sda23:40:261
213699221,19cyclictest221ktimersoftd/122:20:141
213699221,19cyclictest0-21swapper/100:30:171
213699221,18cyclictest12396-21perl19:30:141
213699221,18cyclictest11027-21cstates21:55:131
213699221,11cyclictest0-21swapper/121:43:021
213699221,10cyclictest11677-21cat19:30:001
212899226,4cyclictest8564-1kworker/u9:122:27:100
212899226,3cyclictest8564-1kworker/u9:100:36:300
212899226,3cyclictest8564-1kworker/u9:100:36:300
212899222,17cyclictest14097-21/usr/sbin/munin22:00:220
212899221,5cyclictest0-21swapper/023:46:050
212899221,4cyclictest0-21swapper/023:36:170
214899219,3cyclictest87350irq/126-enp1s0-19:20:153
214899217,3cyclictest27715-21sed22:30:253
214899212,11cyclictest1172-40dnf00:05:233
2148992114,4cyclictest28769-21unixbench_singl20:05:273
214899211,12cyclictest28436-21/usr/sbin/munin22:35:233
214899211,12cyclictest20087-21ntp_states21:00:223
214899211,12cyclictest20087-21ntp_states21:00:213
214899211,12cyclictest0-21swapper/323:47:173
214899210,8cyclictest41-21ksoftirqd/300:21:333
214199217,3cyclictest87350irq/126-enp1s0-21:05:122
214199215,8cyclictest87350irq/126-enp1s0-20:00:122
214199215,4cyclictest8410-21awk00:25:212
214199212,10cyclictest5227-21/usr/sbin/munin19:15:152
214199211,8cyclictest30011-21/usr/sbin/munin21:25:222
214199211,8cyclictest0-21swapper/219:23:322
214199211,7cyclictest0-21swapper/221:11:452
214199211,5cyclictest0-21swapper/222:35:452
214199211,4cyclictest12465-21strings23:10:202
214199211,4cyclictest0-21swapper/200:12:092
213699213,2cyclictest87550irq/128-enp1s0-20:26:421
213699211,18cyclictest0-21swapper/120:47:301
213699211,17cyclictest22123-21tail19:50:251
213699211,17cyclictest22123-21tail19:50:251
213699210,19cyclictest0-21swapper/120:35:021
212899218,8cyclictest0-21swapper/019:15:140
212899216,9cyclictest0-21swapper/019:50:380
212899216,9cyclictest0-21swapper/019:50:380
212899216,8cyclictest0-21swapper/019:55:220
212899211,8cyclictest0-21swapper/023:05:590
2128992118,2cyclictest0-21swapper/021:25:150
2128992117,3cyclictest0-21swapper/021:55:250
2128992117,3cyclictest0-21swapper/019:25:210
2128992117,2cyclictest8891-21grep00:25:280
212899211,11cyclictest28785-21hddtemp_smartct23:55:130
214899208,4cyclictest0-21swapper/322:58:233
214899208,3cyclictest87350irq/126-enp1s0-19:35:163
214899205,6cyclictest0-21swapper/300:25:493
214899204,7cyclictest0-21swapper/321:59:363
214899203,7cyclictest0-21swapper/321:11:563
214899201,5cyclictest0-21swapper/323:02:253
214899201,5cyclictest0-21swapper/321:45:303
214899201,4cyclictest0-21swapper/323:52:273
214899201,4cyclictest0-21swapper/322:26:443
214899201,11cyclictest21230-21/usr/sbin/munin21:05:123
214899201,11cyclictest10230-21/usr/sbin/munin20:40:143
214899201,11cyclictest0-21swapper/323:40:063
214899201,11cyclictest0-21swapper/323:27:173
214899201,11cyclictest0-21swapper/322:47:483
214899201,11cyclictest0-21swapper/321:36:163
214899201,11cyclictest0-21swapper/321:23:033
214899201,11cyclictest0-21swapper/321:17:263
214899201,10cyclictest0-21swapper/323:30:263
214899201,10cyclictest0-21swapper/323:15:023
214899200,9cyclictest0-21swapper/322:11:573
214899200,9cyclictest0-21swapper/322:11:573
214899200,9cyclictest0-21swapper/321:27:003
214199206,3cyclictest30-21rcuc/220:55:162
214199205,5cyclictest755-21rngd19:27:422
214199201,7cyclictest0-21swapper/219:35:242
214199201,5cyclictest11194-21diskstats21:55:142
214199201,5cyclictest0-21swapper/223:48:332
214199201,5cyclictest0-21swapper/200:30:472
214199201,11cyclictest808-21snmpd19:44:542
213699201,6cyclictest0-21swapper/120:20:191
2136992016,3cyclictest0-21swapper/120:36:031
2136992016,3cyclictest0-21swapper/119:15:191
2136992015,3cyclictest31387-21awk20:15:021
213699201,17cyclictest28795-21users20:05:271
213699201,17cyclictest0-21swapper/119:41:521
213699201,16cyclictest22863-21/usr/sbin/munin19:55:121
212899202,4cyclictest0-21swapper/023:41:370
2128992018,1cyclictest0-21swapper/019:35:250
2128992016,3cyclictest0-21swapper/022:55:240
212899201,5cyclictest0-21swapper/021:50:130
2128992014,3cyclictest23380-21grep22:20:260
212899201,16cyclictest9021-21strings20:35:200
212899201,16cyclictest25988-21strings20:00:200
212899201,12cyclictest0-21swapper/021:00:140
212899201,12cyclictest0-21swapper/021:00:140
212899201,10cyclictest5383-21sshd20:27:190
212899201,10cyclictest0-21swapper/000:16:130
146272202,14sleep314732-21cut20:50:123
214899198,3cyclictest3639-21/usr/sbin/munin20:25:123
214899193,9cyclictest0-21swapper/300:37:563
214899193,9cyclictest0-21swapper/300:37:563
214899192,2cyclictest12050irq/123-ahci[0020:15:023
214899191,7cyclictest24466-21sh23:40:013
214899191,7cyclictest11593-21munin-plugin-st19:30:013
214899191,11cyclictest0-21swapper/300:17:443
214199194,4cyclictest87350irq/126-enp1s0-19:45:162
214199192,14cyclictest2915-21grep20:20:262
214199191,6cyclictest23649-21grep19:55:192
214199191,6cyclictest15797-21ps20:50:232
214199191,6cyclictest13522-21kworker/u8:120:05:422
214199191,5cyclictest6829-21needreboot20:30:212
214199191,5cyclictest32-21ksoftirqd/220:10:252
214199191,5cyclictest12096-21cstates19:30:122
214199191,5cyclictest0-21swapper/221:01:112
214199191,5cyclictest0-21swapper/221:01:112
2136991916,2cyclictest0-21swapper/119:11:411
213699191,4cyclictest18576-21sshd20:57:221
213699191,16cyclictest0-21swapper/119:35:041
212899195,9cyclictest0-21swapper/019:30:180
212899194,8cyclictest6856-21users00:21:330
212899191,7cyclictest18403-21df23:25:120
212899191,7cyclictest0-21swapper/023:24:370
212899191,7cyclictest0-21swapper/023:24:370
212899191,6cyclictest5835-21/usr/sbin/munin20:30:190
212899191,6cyclictest0-21swapper/022:46:480
2128991916,2cyclictest0-21swapper/023:00:270
212899191,5cyclictest0-21swapper/023:31:290
212899191,5cyclictest0-21swapper/021:37:350
2128991914,3cyclictest7425-21/usr/sbin/munin19:20:260
2128991914,3cyclictest3597-21smart_sda19:10:240
2128991914,3cyclictest25627-21/usr/sbin/munin21:15:150
2128991914,2cyclictest27206-21/usr/sbin/munin20:05:190
212899191,3cyclictest87550irq/128-enp1s0-00:04:240
2128991913,3cyclictest20886-21find22:15:240
212899191,15cyclictest7962-21sed21:45:290
212899191,12cyclictest0-21swapper/023:17:500
212899191,12cyclictest0-21swapper/000:10:400
212899191,10cyclictest0-21swapper/022:35:240
212899190,2cyclictest87550irq/128-enp1s0-00:35:020
13192190,5sleep30-21swapper/320:20:023
214899182,10cyclictest0-21swapper/322:20:443
214899181,9cyclictest808-21snmpd19:44:543
214899181,6cyclictest579-21systemd-journal19:50:013
214899181,6cyclictest0-21swapper/322:19:303
214199184,3cyclictest87350irq/126-enp1s0-20:40:162
212899183,7cyclictest0-21swapper/020:45:070
212899182,10cyclictest0-21swapper/023:51:560
212899181,8cyclictest22867-21kworker/0:321:12:550
212899181,6cyclictest0-21swapper/023:10:300
212899181,5cyclictest0-21swapper/021:05:200
212899181,15cyclictest0-21swapper/019:45:040
2128991811,4cyclictest18519-21ntpq22:10:200
2128991811,4cyclictest18519-21ntpq22:10:200
212899181,14cyclictest1417-21/usr/sbin/munin20:20:200
212899180,11cyclictest0-21swapper/000:09:560
214899174,4cyclictest22798-21sshd19:55:103
214899172,3cyclictest13293-21grep19:30:253
214899171,7cyclictest0-21swapper/320:30:083
214899171,6cyclictest579-21systemd-journal20:50:003
2148991712,3cyclictest18792-21turbostat21:00:003
214899171,14cyclictest6458-21find19:15:233
214899170,8cyclictest0-21swapper/320:20:173
214199174,3cyclictest87650irq/129-enp1s0-19:10:182
212899171,13cyclictest15844-21tail22:05:170
10542170,2sleep00-21swapper/020:18:310
214899165,3cyclictest87350irq/126-enp1s0-19:50:133
214899165,3cyclictest87350irq/126-enp1s0-19:50:133
214899161,6cyclictest0-21swapper/320:00:213
214899161,5cyclictest9993-21latency_hist20:40:003
212899161,2cyclictest87550irq/128-enp1s0-20:40:160
212899161,2cyclictest0-21swapper/020:11:410
212899151,12cyclictest16522-21sshd20:54:110
2128991510,3cyclictest16454-21expr19:40:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional