You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-22 - 09:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot3s.osadl.org (updated Tue Sep 22, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
72032236166,22sleep10-21swapper/119:09:501
72162202164,26sleep20-21swapper/219:09:582
71132202165,25sleep00-21swapper/019:08:400
52532201165,24sleep30-21swapper/319:05:043
3066321810,4sleep2730999cyclictest20:00:192
245942630,7sleep1221ktimersoftd/122:15:011
7309995148,2cyclictest0-21swapper/223:08:012
731399351,14cyclictest0-21swapper/322:00:133
730399351,14cyclictest0-21swapper/123:12:041
729299357,13cyclictest300-21munin-run21:20:010
731399341,26cyclictest0-21swapper/322:35:263
730399341,9cyclictest0-21swapper/122:54:431
730399341,9cyclictest0-21swapper/122:54:431
729299345,14cyclictest1644-21munin-run23:50:000
7292993410,11cyclictest20205-21munin-run22:05:010
731399331,16cyclictest0-21swapper/321:49:513
730399331,14cyclictest0-21swapper/121:55:351
730399331,12cyclictest0-21swapper/122:41:551
730399331,12cyclictest0-21swapper/122:01:471
729299339,11cyclictest29005-21idleruntime-cro22:25:010
731399321,23cyclictest0-21swapper/300:20:283
731399321,16cyclictest0-21swapper/319:48:133
730399321,14cyclictest0-21swapper/122:22:511
730399321,13cyclictest0-21swapper/119:39:361
730399321,13cyclictest0-21swapper/100:06:571
729299328,11cyclictest9832-21awk22:55:020
729299328,11cyclictest9832-21awk22:55:020
730399311,23cyclictest0-21swapper/100:18:331
730399311,14cyclictest24156-21memory23:25:201
730399311,14cyclictest0-21swapper/120:42:311
730399311,11cyclictest0-21swapper/123:05:561
729299311,15cyclictest23358-21munin-run19:45:010
731399301,27cyclictest0-21swapper/300:01:503
731399301,23cyclictest0-21swapper/323:12:333
731399300,23cyclictest0-21swapper/322:53:253
731399300,23cyclictest0-21swapper/322:53:253
731399300,14cyclictest0-21swapper/320:56:333
731399300,12cyclictest0-21swapper/300:30:143
730399301,6cyclictest0-21swapper/100:29:011
730399301,12cyclictest0-21swapper/123:23:561
730399301,11cyclictest0-21swapper/123:53:191
731399291,23cyclictest0-21swapper/319:12:573
731399291,14cyclictest0-21swapper/321:15:333
731399291,14cyclictest0-21swapper/319:58:333
731399291,13cyclictest0-21swapper/319:50:253
731399291,12cyclictest0-21swapper/321:03:013
731399291,11cyclictest0-21swapper/320:43:023
731399290,19cyclictest0-21swapper/323:59:343
731399290,19cyclictest0-21swapper/323:59:333
731399290,14cyclictest0-21swapper/320:34:143
731399290,14cyclictest0-21swapper/320:34:143
730399291,5cyclictest0-21swapper/122:45:351
730399291,5cyclictest0-21swapper/121:14:081
730399291,12cyclictest0-21swapper/123:15:121
730399291,12cyclictest0-21swapper/100:12:491
730399291,11cyclictest0-21swapper/122:29:351
730399291,10cyclictest0-21swapper/119:56:041
730399291,10cyclictest0-21swapper/119:33:481
729299296,11cyclictest29618-21awk23:40:000
7292992910,16cyclictest15798-21munin-run21:55:010
71752290,4sleep0729299cyclictest22:45:270
55542290,4sleep3731399cyclictest22:45:023
731399281,18cyclictest0-21swapper/322:32:223
731399281,14cyclictest0-21swapper/322:22:383
731399281,14cyclictest0-21swapper/320:13:333
731399281,13cyclictest0-21swapper/322:27:103
731399281,13cyclictest0-21swapper/300:14:363
731399281,12cyclictest0-21swapper/323:16:093
730399281,9cyclictest0-21swapper/119:16:431
730399281,8cyclictest0-21swapper/121:31:201
730399281,6cyclictest0-21swapper/121:40:321
730399281,6cyclictest0-21swapper/121:40:321
730399281,17cyclictest9478-40dnf19:15:021
730399281,12cyclictest0-21swapper/121:48:521
730399281,12cyclictest0-21swapper/119:52:001
730399281,11cyclictest0-21swapper/123:47:031
730399281,11cyclictest0-21swapper/123:00:191
730399281,11cyclictest0-21swapper/122:58:551
730399281,11cyclictest0-21swapper/120:23:351
730399281,10cyclictest0-21swapper/123:57:391
730399281,10cyclictest0-21swapper/123:57:391
730399280,11cyclictest0-21swapper/121:52:511
730399280,11cyclictest0-21swapper/121:00:071
729299281,14cyclictest14231-21munin-run23:05:020
731399279,3cyclictest0-21swapper/322:59:293
731399279,3cyclictest0-21swapper/320:37:543
731399278,3cyclictest0-21swapper/321:22:253
7313992721,3cyclictest6549-21/usr/sbin/munin22:45:203
7313992718,3cyclictest0-21swapper/323:08:493
731399271,12cyclictest0-21swapper/300:25:483
731399271,11cyclictest0-21swapper/323:28:093
731399271,10cyclictest0-21swapper/320:28:343
731399270,13cyclictest0-21swapper/321:28:093
730399271,12cyclictest0-21swapper/121:35:241
730399271,11cyclictest0-21swapper/122:33:231
730399271,11cyclictest0-21swapper/100:36:091
730399271,10cyclictest0-21swapper/100:34:571
731399269,3cyclictest23360-21perf19:45:013
731399261,13cyclictest0-21swapper/323:01:253
731399261,12cyclictest0-21swapper/323:46:533
731399261,11cyclictest13772-21ls00:15:203
731399261,11cyclictest0-21swapper/323:50:093
731399261,11cyclictest0-21swapper/323:43:573
731399261,10cyclictest0-21swapper/322:10:063
731399261,10cyclictest0-21swapper/320:09:333
731399260,24cyclictest0-21swapper/323:20:453
731399260,12cyclictest0-21swapper/320:45:503
730399261,9cyclictest0-21swapper/122:05:551
7303992614,2cyclictest0-21swapper/121:20:201
730399261,18cyclictest0-21swapper/120:58:311
730399261,15cyclictest0-21swapper/100:20:091
730399261,11cyclictest0-21swapper/122:16:231
730399261,10cyclictest0-21swapper/120:04:321
730399260,9cyclictest0-21swapper/120:36:351
287632260,5sleep328765-21pmu-power23:35:213
731399258,3cyclictest0-21swapper/319:16:423
731399257,3cyclictest0-21swapper/319:21:413
731399257,3cyclictest0-21swapper/319:21:413
731399257,3cyclictest0-21swapper/300:06:563
731399251,14cyclictest0-21swapper/321:12:253
731399251,12cyclictest28367-21diskmemload21:34:213
731399251,11cyclictest0-21swapper/321:36:053
731399251,10cyclictest0-21swapper/321:40:533
731399251,10cyclictest0-21swapper/321:40:533
731399251,10cyclictest0-21swapper/319:33:293
731399251,10cyclictest0-21swapper/300:35:123
731399250,11cyclictest0-21swapper/322:07:263
7309992512,3cyclictest87350irq/126-enp1s0-23:55:262
7309992512,3cyclictest87350irq/126-enp1s0-23:55:262
730999250,10cyclictest0-21swapper/200:21:002
730399258,8cyclictest24880-21kworker/u8:219:45:061
730399251,17cyclictest0-21swapper/122:39:111
730399251,15cyclictest0-21swapper/123:36:351
730399251,10cyclictest0-21swapper/123:40:591
730399251,10cyclictest0-21swapper/121:26:321
730399251,10cyclictest0-21swapper/120:05:191
730399251,10cyclictest0-21swapper/100:02:311
731399246,5cyclictest0-21swapper/319:28:313
731399241,12cyclictest0-21swapper/321:58:053
731399241,12cyclictest0-21swapper/320:24:293
731399240,12cyclictest0-21swapper/322:17:143
731399240,10cyclictest0-21swapper/321:08:093
730999242,4cyclictest23358-21munin-run19:45:012
730999242,14cyclictest12050irq/123-ahci[0022:10:022
7309992410,3cyclictest0-21swapper/220:35:332
7303992417,4cyclictest12984-21munin-run20:35:011
7303992417,4cyclictest12984-21munin-run20:35:011
730399241,11cyclictest0-21swapper/120:45:151
730399241,10cyclictest0-21swapper/121:19:321
730399241,10cyclictest0-21swapper/120:28:031
731399238,3cyclictest0-21swapper/320:01:333
731399231,15cyclictest0-21swapper/319:38:453
7309992312,3cyclictest12050irq/123-ahci[0019:15:012
730399231,15cyclictest0-21swapper/120:52:151
730399231,11cyclictest0-21swapper/120:14:231
729299235,10cyclictest21688-21smart_sda22:05:240
731399227,3cyclictest0-21swapper/320:52:253
731399225,3cyclictest14897-21ntpq21:50:213
731399220,11cyclictest0-21swapper/320:19:093
730999225,9cyclictest32-21ksoftirqd/200:13:002
730399221,12cyclictest22130-21kworker/u8:019:20:311
730399221,12cyclictest22130-21kworker/u8:019:20:311
730399221,10cyclictest0-21swapper/121:05:481
730399221,10cyclictest0-21swapper/120:15:151
729299227,5cyclictest0-21swapper/022:58:540
729299224,16cyclictest26811-21perf22:20:010
7292992210,3cyclictest11-21rcuc/019:30:240
729299220,11cyclictest0-21swapper/023:12:500
729299220,11cyclictest0-21swapper/023:06:380
212222220,1sleep00-21swapper/019:40:020
731399211,11cyclictest0-21swapper/323:31:553
730999219,3cyclictest87350irq/126-enp1s0-21:10:012
730999212,3cyclictest0-21swapper/200:30:052
730999212,16cyclictest20320-21head19:35:232
730999211,7cyclictest87650irq/129-enp1s0-21:10:252
730999211,13cyclictest0-21swapper/222:20:142
730399216,3cyclictest0-21swapper/119:41:201
730399211,17cyclictest15054-21cstates19:25:141
729299218,5cyclictest0-21swapper/000:00:210
729299216,6cyclictest0-21swapper/000:07:560
729299211,12cyclictest12589-21/usr/sbin/munin19:20:180
729299211,12cyclictest12589-21/usr/sbin/munin19:20:170
729299211,12cyclictest1229-21/usr/sbin/munin22:35:150
729299210,10cyclictest0-21swapper/023:57:490
729299210,10cyclictest0-21swapper/023:57:490
729299210,10cyclictest0-21swapper/021:26:040
729299210,10cyclictest0-21swapper/000:12:440
730999206,5cyclictest87350irq/126-enp1s0-23:40:142
730999205,3cyclictest0-21swapper/200:08:122
730999201,8cyclictest0-21swapper/223:50:192
7309992014,4cyclictest20608-21irqstats20:50:172
730399201,11cyclictest0-21swapper/123:32:161
729299209,3cyclictest8042-21irqstats21:35:160
729299209,3cyclictest535-21/usr/sbin/munin21:20:250
729299203,9cyclictest0-21swapper/020:30:290
729299203,9cyclictest0-21swapper/020:30:290
729299201,5cyclictest0-21swapper/021:11:200
729299201,11cyclictest27974-21/usr/sbin/munin19:55:250
729299201,11cyclictest16449-21timerandwakeup00:20:250
729299201,11cyclictest16021-21/usr/sbin/munin21:55:230
729299201,10cyclictest9981-21fschecks_time21:40:150
729299201,10cyclictest9981-21fschecks_time21:40:150
729299201,10cyclictest24751-21tail23:25:260
729299201,10cyclictest0-21swapper/021:33:160
730999197,5cyclictest87650irq/129-enp1s0-23:45:162
730999194,5cyclictest16540-21tune2fs21:55:162
730999194,4cyclictest32402-40dnf21:15:332
730999193,4cyclictest14638-40dnf00:17:262
730999192,5cyclictest32-21ksoftirqd/223:25:212
730999191,8cyclictest0-21swapper/221:30:192
730999191,4cyclictest0-21swapper/222:33:562
729299199,3cyclictest6611-21/usr/sbin/munin20:20:120
729299196,3cyclictest0-21swapper/023:50:390
729299196,3cyclictest0-21swapper/022:13:370
729299191,7cyclictest87350irq/126-enp1s0-00:25:020
7292991914,3cyclictest9478-40dnf19:15:000
7292991913,3cyclictest19563-21idleruntime-cro20:50:010
7292991912,4cyclictest27017-21fgrep19:50:220
729299191,13cyclictest0-21swapper/019:45:240
729299191,11cyclictest12284-21NetworkManager19:19:070
729299191,10cyclictest21682-21/usr/sbin/munin00:35:120
729299191,10cyclictest0-21swapper/023:21:570
730999187,4cyclictest87650irq/129-enp1s0-21:20:162
730999186,3cyclictest12050irq/123-ahci[0023:00:012
730999184,3cyclictest32-21ksoftirqd/222:00:192
730999183,6cyclictest0-21swapper/222:45:242
730999182,6cyclictest27027-21users23:30:272
730999182,6cyclictest0-21swapper/221:40:132
730999182,6cyclictest0-21swapper/221:40:132
730999182,5cyclictest8564-1kworker/u9:100:39:272
730999181,8cyclictest909-21sssd_nss19:55:182
730999181,7cyclictest30030-21irqstats22:25:182
730999181,7cyclictest0-21swapper/223:24:222
730999181,5cyclictest0-21swapper/221:35:112
7309991814,3cyclictest32-21ksoftirqd/220:30:012
729299186,3cyclictest0-21swapper/022:26:290
729299182,7cyclictest32487-21ntp_states22:30:250
729299181,8cyclictest28381-21perf21:10:020
729299181,6cyclictest0-21swapper/022:41:580
7292991812,4cyclictest0-21swapper/023:15:590
7292991812,3cyclictest4193-21perf20:15:000
7292991811,4cyclictest13155-21grep21:45:250
729299181,11cyclictest0-21swapper/000:30:370
729299181,11cyclictest0-21swapper/000:15:220
51792181,2sleep20-21swapper/222:42:082
730999176,3cyclictest87350irq/126-enp1s0-20:30:142
730999176,3cyclictest87350irq/126-enp1s0-20:30:132
730999174,3cyclictest30-21rcuc/222:19:542
730999173,7cyclictest0-21swapper/223:16:212
730999173,7cyclictest0-21swapper/222:37:422
730999173,4cyclictest0-21swapper/222:52:022
730999173,4cyclictest0-21swapper/222:52:022
730999172,8cyclictest87350irq/126-enp1s0-23:00:122
730999172,5cyclictest2218-21/usr/sbin/munin20:10:172
730999172,11cyclictest32-21ksoftirqd/219:25:202
730999171,7cyclictest0-21swapper/222:12:232
730999171,7cyclictest0-21swapper/221:29:372
730999171,6cyclictest23581-21/usr/sbin/munin19:45:142
730999171,6cyclictest0-21swapper/223:37:032
730999171,6cyclictest0-21swapper/220:15:242
730999171,5cyclictest30-21rcuc/220:05:142
7309991711,3cyclictest22621-21grep20:55:162
729299177,3cyclictest808-21snmpd20:25:010
729299174,7cyclictest29482-21kworker/u8:223:40:560
729299174,4cyclictest1-21systemd19:30:020
7292991712,3cyclictest22207-21expr20:55:140
7292991712,3cyclictest16105-21seq20:40:170
729299171,10cyclictest0-21swapper/020:15:200
729299170,5cyclictest28367-21diskmemload23:30:140
730999161,9cyclictest0-21swapper/221:50:192
730999161,7cyclictest16061-21kworker/u8:123:10:252
730999161,7cyclictest0-21swapper/200:27:122
730999161,6cyclictest6611-21/usr/sbin/munin20:20:262
730999161,6cyclictest0-21swapper/219:15:212
730999161,6cyclictest0-21swapper/200:00:292
730999161,3cyclictest0-21swapper/221:00:202
7309991611,3cyclictest18559-21cat20:45:192
7309991611,3cyclictest13637-21cut21:50:022
730999161,12cyclictest27320-21grep19:50:242
7309991610,3cyclictest17472-21grep20:45:012
729299164,3cyclictest31769-21/usr/sbin/munin20:00:290
729299161,7cyclictest808-21snmpd20:50:060
729299161,6cyclictest0-21swapper/020:05:190
7292991610,3cyclictest14353-21awk20:35:220
182162161,2sleep20-21swapper/219:30:252
7309991511,3cyclictest32-21ksoftirqd/219:20:202
7309991511,3cyclictest32-21ksoftirqd/219:20:202
729299152,10cyclictest25750-21grep21:00:280
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional