You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-13 - 16:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Sat Jun 13, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
834323340,4sleep341-21ksoftirqd/307:05:273
86812236162,25sleep00-21swapper/007:09:410
84782234167,55sleep10-21swapper/107:07:041
83822202167,23sleep20-21swapper/207:05:482
1334721440,4sleep0878999cyclictest08:30:180
162722560,3sleep1879799cyclictest11:05:141
167962490,3sleep19-21rcu_preempt09:50:251
266552460,2sleep00-21swapper/007:45:270
202172380,4sleep020219-21head08:45:230
8789993710,13cyclictest9737-21munin-run12:05:000
880999355,20cyclictest17034-21fschecks_time12:20:153
878999345,13cyclictest32067-21awk09:15:010
184642340,5sleep3880999cyclictest11:10:133
880999334,10cyclictest29862-21perf09:10:003
878999326,12cyclictest29856-21munin-run09:10:010
878999321,19cyclictest1692-21kworker/0:109:30:200
878999321,19cyclictest1692-21kworker/0:109:30:190
879799311,12cyclictest30073-21kworker/u8:009:36:441
878999318,11cyclictest22656-21perf07:40:020
878999301,13cyclictest30472-21munin-run10:25:010
218932300,4sleep123-21ksoftirqd/110:05:011
8809992915,6cyclictest32160-21/usr/sbin/munin10:25:273
878999290,13cyclictest6915-21idleruntime-cro10:45:010
878999290,13cyclictest6915-21idleruntime-cro10:45:010
880999287,13cyclictest23382-21expr12:35:123
880999283,18cyclictest27530-21/usr/sbin/munin10:15:233
879799281,21cyclictest0-21swapper/112:04:011
878999284,11cyclictest17292-21awk09:55:010
880999277,9cyclictest3179-21ntp_offset09:20:213
880999273,13cyclictest26394-40dnf11:26:223
880999271,15cyclictest0-21swapper/312:26:363
879799278,9cyclictest8564-1kworker/u9:107:17:381
879799271,18cyclictest0-21swapper/110:45:441
878999271,11cyclictest23685-21kworker/0:010:14:280
878999271,10cyclictest22209-21kworker/0:311:30:240
880999267,5cyclictest1133-21/usr/sbin/munin11:45:203
880999262,18cyclictest87350irq/126-enp1s0-11:35:163
8809992615,7cyclictest7158-21smart_sda08:15:243
880299267,4cyclictest3521-21/usr/sbin/munin09:20:262
880299263,7cyclictest20440-40dnf12:26:362
879799263,10cyclictest20433-21kworker/u8:112:30:161
879799261,17cyclictest0-21swapper/110:36:071
879799260,18cyclictest16903-21gltestperf07:25:171
878999262,12cyclictest5336-21idleruntime-cro11:55:000
878999262,12cyclictest5336-21idleruntime-cro11:55:000
880999259,4cyclictest15299-21/usr/sbin/munin12:15:203
880999256,11cyclictest808-21snmpd07:42:413
880999252,10cyclictest18555-21ntp_kernel_pll_09:55:203
880999250,20cyclictest87350irq/126-enp1s0-12:30:133
880999250,18cyclictest5531-21expr10:40:173
880999250,18cyclictest5531-21expr10:40:173
880999250,14cyclictest6369-21kernelversion11:55:173
879799259,4cyclictest11880-21diskstats10:55:141
8797992519,3cyclictest31196-21cut09:10:231
8797992519,3cyclictest2468-21diskstats09:20:151
879799251,6cyclictest0-21swapper/112:10:181
879799251,20cyclictest23882-21munin-run10:10:011
879799251,17cyclictest0-21swapper/110:10:121
879799251,15cyclictest0-21swapper/110:25:011
878999251,9cyclictest14232-21kworker/0:208:35:560
144902250,4sleep230-21rcuc/211:00:162
880999247,5cyclictest808-21snmpd09:03:333
880999246,5cyclictest11359-21cut09:40:153
880999246,3cyclictest13728-21/usr/sbin/munin11:00:273
880999245,10cyclictest26072-21latency_hist10:15:013
880999244,9cyclictest16359-21ntp_kernel_pll_09:50:213
880999242,9cyclictest20326-21/usr/sbin/munin11:15:213
880999242,12cyclictest839-21lldpd09:15:163
8809992414,6cyclictest4964-21smart_sda08:10:243
880999241,13cyclictest0-21swapper/307:35:173
880999240,11cyclictest31653-21df_inode11:40:143
879799245,10cyclictest10303-21cat08:25:111
879799242,12cyclictest5418-40dnf09:25:251
879799241,17cyclictest0-21swapper/110:15:171
879799241,17cyclictest0-21swapper/108:57:471
8797992410,8cyclictest0-21swapper/108:40:331
8789992412,2cyclictest20317-21kworker/0:111:15:440
878999241,11cyclictest15099-21idleruntime-cro09:50:000
880999238,5cyclictest8715-21/usr/sbin/munin09:35:113
880999236,9cyclictest31661-21/usr/sbin/munin08:00:303
880999234,6cyclictest0-21swapper/308:05:123
880999233,8cyclictest13040-21proc_pri07:15:243
880999232,11cyclictest10299-21/usr/sbin/munin08:25:273
880999231,10cyclictest808-21snmpd07:50:143
8802992316,4cyclictest27924-21cut07:50:162
879799239,2cyclictest6601-21ntp_offset11:55:211
879799238,2cyclictest22286-21crond11:20:021
879799231,17cyclictest20433-21kworker/u8:112:36:151
879799231,16cyclictest0-21swapper/111:02:081
8797992311,3cyclictest0-21swapper/110:54:281
879799231,11cyclictest31896-21fw_packets11:40:171
879799231,11cyclictest20453-21sleep07:35:021
879799231,10cyclictest29794-21df07:55:131
879799231,10cyclictest10959-21latency_hist07:15:011
879799231,10cyclictest10959-21latency_hist07:15:001
879799231,10cyclictest0-21swapper/111:23:011
8797992310,3cyclictest18059-21perl09:55:151
878999231,9cyclictest15628-21kworker/0:111:07:240
878999231,6cyclictest0-21swapper/009:37:450
878999231,4cyclictest0-21swapper/010:36:080
8789992311,3cyclictest12722-21kworker/0:110:02:270
880999224,9cyclictest9165-21ls07:10:133
880999224,9cyclictest9165-21ls07:10:133
880999223,16cyclictest12612-21grep08:30:123
880999223,10cyclictest0-21swapper/311:52:233
880999223,10cyclictest0-21swapper/311:52:233
880999221,7cyclictest0-21swapper/309:49:553
8809992215,4cyclictest12664-21grep10:55:223
880999220,9cyclictest0-21swapper/310:00:163
880299226,8cyclictest87350irq/126-enp1s0-09:00:232
880299225,5cyclictest21689-21perf10:05:022
880299221,5cyclictest23631-21kworker/u8:110:20:122
879799228,8cyclictest0-21swapper/111:53:381
879799228,8cyclictest0-21swapper/111:53:381
879799228,3cyclictest31204-21cat11:40:011
879799221,9cyclictest15077-21sh09:50:011
879799221,9cyclictest10068-21sh08:25:001
879799221,6cyclictest0-21swapper/110:33:381
8797992214,2cyclictest0-21swapper/111:25:161
879799221,3cyclictest4381-21smartctl08:10:161
879799221,3cyclictest20147-21timerwakeupswit12:25:251
879799221,19cyclictest0-21swapper/112:24:071
879799221,19cyclictest0-21swapper/107:36:551
879799221,15cyclictest0-21swapper/108:30:031
879799221,10cyclictest28873-21strings09:05:211
879799220,14cyclictest0-21swapper/111:46:161
878999223,11cyclictest18477-21/usr/sbin/munin07:30:180
8789992217,3cyclictest26252-21perl09:00:150
8789992212,6cyclictest5582-21swap09:25:250
8789992210,6cyclictest18457-21kworker/0:012:37:270
880999217,2cyclictest0-21swapper/310:33:513
880999217,11cyclictest6854-21df09:30:143
880999217,11cyclictest6854-21df09:30:133
880999214,9cyclictest30462-21fschecks_count09:10:143
880999213,15cyclictest16279-21/usr/sbin/munin07:25:143
880999213,13cyclictest0-21swapper/309:29:553
880999212,7cyclictest18659-21cstates07:30:133
880999212,12cyclictest2514-21perf10:39:563
880999212,11cyclictest23631-21kworker/u8:110:23:003
880999211,8cyclictest21910-21/usr/sbin/munin10:05:173
880999211,16cyclictest839-21lldpd12:05:203
880999211,15cyclictest0-21swapper/312:00:083
880999211,14cyclictest87350irq/126-enp1s0-11:06:183
880999211,11cyclictest0-21swapper/310:45:173
880299216,3cyclictest12050irq/123-ahci[0007:30:002
880299211,6cyclictest808-21snmpd11:43:272
880299211,6cyclictest0-21swapper/211:48:362
879799218,2cyclictest25083-21egrep07:45:131
879799217,3cyclictest19843-21if_enp2s008:45:171
879799211,8cyclictest0-21swapper/109:40:151
879799211,4cyclictest0-21swapper/112:05:161
8797992112,3cyclictest0-21swapper/108:52:481
879799211,15cyclictest0-21swapper/111:13:091
879799211,14cyclictest0-21swapper/109:01:581
879799211,11cyclictest3479-21munin-run08:10:011
879799211,11cyclictest0-21swapper/109:33:591
879799211,11cyclictest0-21swapper/109:33:591
879799211,10cyclictest0-21swapper/112:17:531
879799211,10cyclictest0-21swapper/110:25:151
8797992110,3cyclictest0-21swapper/109:15:021
879799210,19cyclictest0-21swapper/107:50:211
879799210,15cyclictest0-21swapper/111:33:421
879799210,10cyclictest0-21swapper/108:15:171
879799210,10cyclictest0-21swapper/107:25:011
878999218,3cyclictest2890-21kworker/0:011:59:310
878999217,6cyclictest0-21swapper/011:25:220
878999216,2cyclictest29040-21kworker/0:007:53:150
878999215,2cyclictest31814-21kworker/0:209:19:000
878999212,17cyclictest26650-21kworker/0:011:35:280
878999211,3cyclictest0-21swapper/011:13:340
880999208,2cyclictest0-21swapper/308:40:023
880999207,2cyclictest0-21swapper/308:40:203
880999205,12cyclictest27040-21ls07:50:013
880999205,12cyclictest21277-21/usr/sbin/munin08:50:103
880999204,3cyclictest8430-21chrt08:20:143
880999203,14cyclictest19248-21cpuspeed_turbos08:45:133
880999202,10cyclictest29460-21/usr/sbin/munin07:55:213
880999201,8cyclictest0-21swapper/310:50:253
880999201,12cyclictest29858-21diskmemload12:13:333
880299209,8cyclictest29858-21diskmemload12:05:152
880299206,3cyclictest5750irq/9-acpi12:35:132
880299205,3cyclictest17812-21smart_sda12:20:242
880299205,3cyclictest12050irq/123-ahci[0011:15:012
880299203,4cyclictest16303-21sshd08:35:332
880299201,7cyclictest808-21snmpd07:12:372
880299201,7cyclictest808-21snmpd07:12:372
880299201,7cyclictest0-21swapper/211:58:442
880299201,6cyclictest32664-21fschecks_count09:15:152
880299201,6cyclictest32576-21memory08:00:212
880299201,6cyclictest29110-21gzip09:05:242
880299201,6cyclictest18419-21memory09:55:182
8802992014,4cyclictest4929-21grep09:25:162
8802992014,3cyclictest18635-21cut07:30:132
880299200,5cyclictest10609-21kworker/u8:209:44:332
879799206,3cyclictest32725-21ntp_states08:00:231
879799202,5cyclictest23934-21ntp_states07:40:231
8797992012,2cyclictest0-21swapper/110:41:071
8797992012,2cyclictest0-21swapper/110:41:071
879799201,10cyclictest16864-21cat08:40:011
878999208,11cyclictest0-21swapper/009:25:010
878999207,3cyclictest23685-21kworker/0:010:29:400
878999206,3cyclictest26703-21kworker/0:307:55:140
878999205,3cyclictest25897-21kworker/0:110:15:240
878999205,2cyclictest87450irq/127-enp1s0-07:29:030
878999203,5cyclictest18457-21kworker/0:012:29:030
878999202,10cyclictest21910-21/usr/sbin/munin10:05:260
878999201,4cyclictest0-21swapper/011:42:330
8789992013,4cyclictest7468-21df_inode10:45:130
878999201,17cyclictest13859-21munin-run07:20:010
880999199,7cyclictest15953-21seq07:23:553
880999193,6cyclictest0-21swapper/311:30:293
8809991910,6cyclictest23810-21/usr/sbin/munin08:55:143
8809991910,6cyclictest23145-21/usr/sbin/munin11:20:143
880299196,3cyclictest87350irq/126-enp1s0-11:30:162
880299195,3cyclictest87650irq/129-enp1s0-11:15:172
880299194,5cyclictest802-21sssd_be12:15:272
880299194,3cyclictest7840-21cpuspeed_turbos12:00:122
880299193,4cyclictest8911-21irqstats08:20:182
880299192,6cyclictest8740-21cpu09:35:122
880299191,7cyclictest0-21swapper/210:18:082
880299191,6cyclictest0-21swapper/211:05:072
880299191,6cyclictest0-21swapper/208:40:152
880299191,5cyclictest30032-21memory11:35:192
880299191,5cyclictest0-21swapper/210:41:562
880299191,5cyclictest0-21swapper/210:41:562
880299191,5cyclictest0-21swapper/209:31:082
880299191,5cyclictest0-21swapper/209:31:082
880299190,6cyclictest16601-21switchtime09:50:242
878999199,3cyclictest10915-21/usr/sbin/munin09:40:150
878999197,3cyclictest18457-21kworker/0:012:32:070
878999196,3cyclictest13368-21kworker/0:211:00:360
878999192,9cyclictest29040-21kworker/0:008:05:180
8789991913,3cyclictest10838-21grep10:50:260
8789991911,3cyclictest31661-21/usr/sbin/munin08:00:240
878999191,10cyclictest8094-21/usr/sbin/munin08:20:240
159222191,2sleep00-21swapper/007:23:330
880299185,3cyclictest87350irq/126-enp1s0-10:35:162
880299185,3cyclictest87350irq/126-enp1s0-10:30:112
880299185,3cyclictest87350irq/126-enp1s0-10:25:172
880299185,3cyclictest87350irq/126-enp1s0-10:05:252
880299185,3cyclictest12050irq/123-ahci[0010:15:012
880299181,7cyclictest0-21swapper/209:47:032
880299181,6cyclictest24719-21/usr/sbin/munin11:25:142
880299181,6cyclictest20254-21ps08:45:222
880299181,5cyclictest9671-21df10:50:132
880299181,5cyclictest24646-21ps08:55:222
880299181,5cyclictest22521-21/usr/sbin/munin11:20:172
880299181,5cyclictest11442-21pmu-power08:25:232
880299181,4cyclictest0-21swapper/212:30:162
8802991812,3cyclictest13105-21unixbench_singl10:55:242
878999185,2cyclictest87550irq/128-enp1s0-10:57:240
878999184,3cyclictest87350irq/126-enp1s0-12:05:120
878999181,9cyclictest808-21snmpd09:58:070
878999181,9cyclictest31276-21kworker/0:108:11:260
8789991813,2cyclictest19593-21kworker/0:107:11:540
8789991813,2cyclictest19593-21kworker/0:107:11:540
878999181,2cyclictest0-21swapper/012:10:210
8789991812,3cyclictest17497-21gltestperf08:40:180
878999181,15cyclictest0-21swapper/012:21:240
878999180,10cyclictest0-21swapper/008:28:340
880299176,2cyclictest30-21rcuc/207:35:182
880299173,3cyclictest87350irq/126-enp1s0-08:10:142
880299173,3cyclictest87350irq/126-enp1s0-08:05:122
880299172,3cyclictest14108-21sshd08:30:332
880299171,8cyclictest0-21swapper/207:17:472
880299171,5cyclictest4067-21basename11:50:152
880299171,5cyclictest4067-21basename11:50:152
880299171,4cyclictest7467-21df10:45:122
880299171,4cyclictest6708-21irqstats08:15:172
880299171,4cyclictest31546-21unixbench-2d09:10:252
880299171,4cyclictest11699-21kworker/u8:212:10:472
8802991712,3cyclictest25915-21cat07:45:192
8802991711,3cyclictest21277-21/usr/sbin/munin08:50:142
878999175,2cyclictest87450irq/127-enp1s0-07:40:070
878999173,3cyclictest2183-21kworker/0:110:33:280
878999171,8cyclictest87550irq/128-enp1s0-08:53:350
878999171,2cyclictest17860-21kworker/0:211:24:280
880299162,3cyclictest87350irq/126-enp1s0-07:40:192
8802991612,3cyclictest0-21swapper/207:23:222
878999169,4cyclictest2328-21wc11:45:240
878999162,2cyclictest87550irq/128-enp1s0-08:57:000
878999161,5cyclictest0-21swapper/012:15:250
878999161,14cyclictest0-21swapper/008:19:180
308002160,4sleep2880299cyclictest07:55:252
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional