You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-21 - 23:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Fri Aug 21, 2026 12:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
70282240165,24sleep10-21swapper/107:09:041
67992237163,24sleep20-21swapper/207:06:072
51402235164,58sleep30-21swapper/307:05:063
69352223151,59sleep00-21swapper/007:07:540
938921310,5sleep0717899cyclictest08:25:150
1316521290,3sleep30-21swapper/312:15:143
49572770,4sleep00-21swapper/008:15:160
124902580,3sleep1718699cyclictest07:20:111
301202510,3sleep330122-21fschecks_count11:40:153
272922500,2sleep327291-21idleruntime-cro11:35:013
202892430,1sleep10-21swapper/111:15:241
96272400,1sleep30-21swapper/309:40:153
7199993412,7cyclictest87350irq/126-enp1s0-09:27:583
718699342,21cyclictest0-21swapper/110:04:411
123562320,3sleep00-21swapper/008:30:250
717899302,16cyclictest16926-21munin-run12:25:020
719999290,13cyclictest0-21swapper/310:50:093
718699292,12cyclictest16652-21munin-run07:30:011
717899296,11cyclictest15694-21idleruntime-cro09:55:010
7199992715,3cyclictest808-21snmpd07:37:073
719999270,7cyclictest0-21swapper/311:10:343
718699271,6cyclictest0-21swapper/110:53:201
719999268,13cyclictest0-21swapper/312:01:503
719999262,10cyclictest1111-21/usr/sbin/munin10:35:253
719999262,10cyclictest1111-21/usr/sbin/munin10:35:243
719299261,22cyclictest0-21swapper/211:13:052
718699269,3cyclictest8479-21awk08:25:011
718699261,10cyclictest4055-21cpuspeed_turbos11:55:151
292322260,4sleep121-21rcuc/110:25:121
719999252,3cyclictest41-21ksoftirqd/309:35:153
719999252,11cyclictest0-21swapper/312:20:233
719999252,10cyclictest28477-21/usr/sbin/munin09:10:133
7186992513,3cyclictest0-21swapper/112:13:501
717899257,15cyclictest18845-21munin-run07:35:020
717899256,11cyclictest0-21swapper/010:50:230
717899251,12cyclictest16315-21basename11:10:000
326982250,4sleep011-21rcuc/009:20:010
719999241,21cyclictest0-21swapper/311:50:453
719999241,21cyclictest0-21swapper/311:50:453
719999241,11cyclictest15903-21/usr/sbin/munin09:55:223
719999241,11cyclictest0-21swapper/308:51:393
719999240,9cyclictest0-21swapper/310:44:333
719999240,11cyclictest0-21swapper/310:27:593
719299245,6cyclictest0-21swapper/209:41:442
719299242,6cyclictest4506-40dnf09:27:592
719299241,4cyclictest0-21swapper/209:15:172
719299241,3cyclictest0-21swapper/211:55:272
7186992418,3cyclictest6591-21ntp_states10:45:211
7186992412,3cyclictest0-21swapper/110:15:561
7186992410,3cyclictest12859-21latency_hist08:35:021
718699240,5cyclictest0-21swapper/111:32:191
7178992417,2cyclictest0-21swapper/012:30:160
717899241,6cyclictest0-21swapper/010:16:270
717899241,10cyclictest3702-21strings09:25:210
719999237,3cyclictest233702cat11:25:133
719999231,7cyclictest0-21swapper/310:45:213
719999231,10cyclictest14671-21/usr/sbin/munin07:25:213
719299231,8cyclictest0-21swapper/212:29:522
719299231,7cyclictest0-21swapper/210:42:192
719299231,7cyclictest0-21swapper/210:20:212
719299231,6cyclictest0-21swapper/212:39:082
719299231,6cyclictest0-21swapper/210:49:202
719299231,4cyclictest0-21swapper/211:03:072
719299231,4cyclictest0-21swapper/209:11:462
719299231,3cyclictest0-21swapper/211:15:412
718699232,3cyclictest19050-40dnf12:29:241
718699232,18cyclictest1508-21munin-run11:50:001
7186992312,2cyclictest0-21swapper/108:38:141
7186992312,2cyclictest0-21swapper/107:56:371
718699231,19cyclictest2045-21memory10:35:181
718699231,19cyclictest2045-21memory10:35:181
7186992310,3cyclictest0-21swapper/109:15:111
717899238,3cyclictest6513-21grep10:45:190
7178992316,2cyclictest0-21swapper/009:30:150
717899231,4cyclictest0-21swapper/011:14:260
717899230,10cyclictest0-21swapper/009:05:210
719999221,8cyclictest24052-21grep10:10:243
719999221,6cyclictest0-21swapper/312:13:053
719999221,12cyclictest0-21swapper/309:30:233
719999220,9cyclictest0-21swapper/312:09:553
719299228,8cyclictest0-21swapper/209:50:202
719299226,3cyclictest3725-21awk11:55:012
719299226,3cyclictest3725-21awk11:55:012
719299224,4cyclictest29438-21df_inode10:25:132
719299221,8cyclictest0-21swapper/212:11:332
719299221,7cyclictest0-21swapper/212:32:462
719299221,6cyclictest10268-21df_inode10:55:142
719299221,6cyclictest0-21swapper/211:35:022
719299221,3cyclictest0-21swapper/212:17:382
719299221,3cyclictest0-21swapper/210:50:532
718699228,3cyclictest87550irq/128-enp1s0-11:43:021
718699228,3cyclictest30399-21diskstats08:00:141
718699223,16cyclictest14008-21smart_sda12:15:231
7186992216,3cyclictest27626-21latency_hist07:55:011
7186992214,3cyclictest0-21swapper/109:14:291
718699221,19cyclictest0-21swapper/107:50:011
718699221,10cyclictest0-21swapper/112:08:171
718699221,10cyclictest0-21swapper/110:58:221
718699221,10cyclictest0-21swapper/109:50:191
718699220,6cyclictest0-21swapper/109:45:261
717899221,7cyclictest0-21swapper/009:21:000
717899221,6cyclictest30605-21sed10:25:250
717899221,6cyclictest0-21swapper/012:39:420
7178992214,2cyclictest0-21swapper/010:35:210
7178992214,2cyclictest0-21swapper/010:35:210
717899220,9cyclictest0-21swapper/007:40:170
717899220,10cyclictest0-21swapper/008:10:200
719999216,3cyclictest4421-21fschecks_time11:55:183
719999214,5cyclictest0-21swapper/311:15:473
719999212,5cyclictest0-21swapper/310:07:433
719999211,11cyclictest0-21swapper/310:20:213
719999210,13cyclictest0-21swapper/311:47:253
719299216,8cyclictest8356-21sshd08:23:452
719299215,4cyclictest21418-21sed10:05:232
719299211,7cyclictest827-21runrttasks11:05:232
719299211,6cyclictest4339-21kworker/u8:109:45:112
719299211,6cyclictest0-21swapper/212:04:502
719299211,5cyclictest0-21swapper/212:08:222
719299211,4cyclictest0-21swapper/209:33:582
719299211,3cyclictest0-21swapper/210:01:422
719299211,3cyclictest0-21swapper/209:37:142
719299211,3cyclictest0-21swapper/209:20:192
719299210,3cyclictest0-21swapper/211:45:152
718699219,2cyclictest14346-21cpu11:05:111
718699217,3cyclictest14109-21munin-run11:05:011
718699214,4cyclictest21264-21sshd12:34:311
718699213,6cyclictest0-21swapper/109:31:341
718699211,8cyclictest0-21swapper/110:40:571
7186992118,2cyclictest0-21swapper/111:50:491
7186992118,2cyclictest0-21swapper/111:50:491
718699211,6cyclictest11342-21ntp_states07:15:211
7186992115,3cyclictest23461-21df_inode11:25:131
718699211,18cyclictest0-21swapper/108:42:501
718699211,18cyclictest0-21swapper/108:13:281
718699210,6cyclictest0-21swapper/112:24:111
718699210,6cyclictest0-21swapper/111:11:151
717899219,2cyclictest24457-21needreboot07:45:200
717899216,13cyclictest11-21rcuc/012:10:170
717899214,3cyclictest87550irq/128-enp1s0-09:55:150
717899213,3cyclictest87550irq/128-enp1s0-12:05:160
717899212,11cyclictest10279-21/usr/sbin/munin07:15:220
717899211,8cyclictest12501-21needreboot09:45:190
717899211,6cyclictest4339-21kworker/u8:109:45:020
717899211,6cyclictest0-21swapper/011:58:420
717899211,6cyclictest0-21swapper/011:04:090
7178992111,3cyclictest20911-21grep10:05:160
717899211,11cyclictest14081-21needreboot08:35:200
719999206,5cyclictest766-21ntpd11:35:203
719999206,5cyclictest10537-21gltestperf10:55:163
719999205,6cyclictest0-21swapper/310:31:043
719999205,4cyclictest8532-21grep07:10:213
719999201,8cyclictest32609-21diskstats08:05:133
719999201,8cyclictest0-21swapper/311:05:513
719999201,6cyclictest0-21swapper/312:34:313
719999201,4cyclictest28248-21diskmemload10:05:023
719999201,11cyclictest0-21swapper/312:26:173
719999201,11cyclictest0-21swapper/309:46:073
719999201,10cyclictest20921-21/usr/sbin/munin11:20:233
719999200,9cyclictest0-21swapper/310:18:523
719999200,6cyclictest5973-21kworker/u8:309:50:033
719299206,4cyclictest13565-21fschecks_time08:35:162
719299202,8cyclictest25958-21smart_sda10:15:232
719299201,7cyclictest0-21swapper/210:11:452
719299201,5cyclictest8107-21idleruntime07:10:162
719299201,5cyclictest25973-21sed11:30:142
719299201,4cyclictest0-21swapper/210:35:562
719299201,4cyclictest0-21swapper/210:35:562
719299201,3cyclictest0-21swapper/209:55:192
719299201,2cyclictest0-21swapper/210:30:112
719299201,10cyclictest0-21swapper/212:21:182
718699205,3cyclictest87550irq/128-enp1s0-08:29:561
718699201,8cyclictest0-21swapper/110:08:291
718699201,6cyclictest0-21swapper/111:36:221
7186992016,3cyclictest0-21swapper/109:27:011
718699201,3cyclictest19462-21perf08:50:021
7186992013,2cyclictest0-21swapper/112:38:251
7186992013,2cyclictest0-21swapper/109:21:481
7186992013,2cyclictest0-21swapper/108:05:581
718699201,17cyclictest0-21swapper/107:30:571
718699201,16cyclictest0-21swapper/110:30:241
718699200,8cyclictest0-21swapper/110:22:081
718699200,8cyclictest0-21swapper/107:35:161
718699200,6cyclictest0-21swapper/111:24:451
718699200,5cyclictest0-21swapper/109:41:511
718699200,3cyclictest24375-21runrttasks10:13:561
718699200,10cyclictest0-21swapper/109:55:241
717899209,3cyclictest21732-21irqstats11:20:180
717899207,2cyclictest0-21swapper/007:35:230
717899204,3cyclictest87550irq/128-enp1s0-11:26:460
717899204,2cyclictest12050irq/123-ahci[0011:20:020
717899202,15cyclictest24734-21cut09:00:150
717899201,7cyclictest28477-21/usr/sbin/munin09:10:140
717899201,6cyclictest11903-21perf11:00:000
717899201,4cyclictest0-21swapper/012:27:070
717899201,4cyclictest0-21swapper/011:40:550
717899201,4cyclictest0-21swapper/011:34:180
717899201,16cyclictest1436-21grep08:05:250
7178992011,2cyclictest22503-21/usr/sbin/munin10:10:150
75172190,2sleep17519-21cpuspeed_turbos07:10:111
719999197,3cyclictest9606-21memory08:25:173
719999194,5cyclictest31344-21grep09:15:173
719999192,14cyclictest2712-21grep08:10:153
719999191,8cyclictest28267-21basename09:10:013
719999191,7cyclictest29820-21crond08:00:003
719999191,7cyclictest0-21swapper/312:36:363
719999191,5cyclictest25429-21latency_hist07:50:003
719999191,4cyclictest808-21snmpd07:42:083
719999191,4cyclictest17483-21/usr/sbin/munin08:45:163
7199991913,4cyclictest18356-21grep07:30:243
719999191,11cyclictest0-21swapper/311:00:223
719999190,8cyclictest0-21swapper/309:20:583
719299191,8cyclictest0-21swapper/207:40:112
719299191,6cyclictest0-21swapper/211:40:122
719299191,4cyclictest0-21swapper/211:20:142
719299191,4cyclictest0-21swapper/207:35:172
719299191,3cyclictest0-21swapper/211:25:042
718699191,7cyclictest0-21swapper/112:00:571
7186991916,2cyclictest0-21swapper/108:18:341
718699191,16cyclictest0-21swapper/107:40:251
718699190,6cyclictest0-21swapper/109:38:441
717899195,3cyclictest87550irq/128-enp1s0-11:50:160
717899195,3cyclictest87550irq/128-enp1s0-11:50:160
717899193,8cyclictest28129-21fgrep10:20:230
717899192,5cyclictest32154-21cat08:05:010
717899192,4cyclictest15018-21diskstats07:25:130
717899191,6cyclictest15447-21kworker/u8:010:00:140
717899191,6cyclictest0-21swapper/012:16:390
717899191,6cyclictest0-21swapper/010:31:510
717899191,4cyclictest9079-21latency_hist09:40:010
5082191,3sleep0524-21ntp_states11:45:220
292732191,2sleep00-21swapper/011:37:280
250672190,2sleep1221ktimersoftd/109:00:201
106142191,2sleep20-21swapper/208:29:322
719999188,3cyclictest808-21snmpd08:17:133
719999185,4cyclictest87650irq/129-enp1s0-08:40:153
719999183,9cyclictest0-21swapper/308:00:143
719999181,5cyclictest11204-21cstates08:30:123
719999181,4cyclictest808-21snmpd08:22:143
719999181,15cyclictest0-21swapper/307:15:093
719999181,10cyclictest839-21lldpd07:21:243
719999180,6cyclictest27623-21crond07:55:003
719299184,4cyclictest23464-21/usr/sbin/munin07:45:152
719299183,5cyclictest30-21rcuc/208:55:242
719299182,5cyclictest9981-21ntp_states07:15:032
719299181,4cyclictest6288-21ls08:20:012
719299181,4cyclictest26279-21/usr/sbin/munin09:05:142
7192991813,3cyclictest18714-21find08:45:222
7192991812,3cyclictest15283-21/usr/sbin/munin08:40:132
719299181,1cyclictest0-21swapper/208:51:392
719299181,10cyclictest774-21avahi-daemon07:32:292
719299181,10cyclictest27275-21taskset07:50:362
718699183,2cyclictest87350irq/126-enp1s0-08:50:141
718699181,5cyclictest21879-21/usr/sbin/munin08:55:251
718699181,16cyclictest0-21swapper/109:05:111
717899185,3cyclictest87350irq/126-enp1s0-08:50:180
717899184,3cyclictest13282-21irqstats07:20:160
717899182,3cyclictest87350irq/126-enp1s0-12:03:250
717899182,3cyclictest4806-21taskset10:40:250
717899180,9cyclictest0-21swapper/007:55:270
318552181,3sleep20-21swapper/208:02:552
719999172,5cyclictest14158-21ntp_states08:35:223
719999171,7cyclictest22706-21kernelversion08:55:193
719999171,6cyclictest24413-21df_inode09:00:143
719299171,5cyclictest0-21swapper/208:05:222
719299171,5cyclictest0-21swapper/207:55:152
719299171,4cyclictest0-21swapper/209:00:182
719299171,2cyclictest0-21swapper/208:30:242
717899174,3cyclictest87550irq/128-enp1s0-08:45:240
717899171,3cyclictest0-21swapper/007:10:120
7178991711,3cyclictest7562-21ntp_states08:20:200
7178991710,2cyclictest0-21swapper/007:50:150
719299161,6cyclictest12473-21/usr/sbin/munin07:20:152
719299161,2cyclictest0-21swapper/208:10:122
7192991611,3cyclictest14671-21/usr/sbin/munin07:25:152
717899163,3cyclictest87350irq/126-enp1s0-08:55:120
717899160,15cyclictest0-21swapper/008:40:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional