You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-19 - 09:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Sat Sep 19, 2026 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
63792236163,24sleep20-21swapper/219:07:482
62662216166,38sleep30-21swapper/319:06:203
65432212161,39sleep10-21swapper/119:09:561
63252202165,24sleep00-21swapper/019:07:060
1978221570,3sleep1663999cyclictest22:05:131
718221370,5sleep3665099cyclictest19:10:133
6646999894,2cyclictest0-21swapper/221:33:092
6646998379,3cyclictest0-21swapper/222:46:352
6646998379,3cyclictest0-21swapper/222:46:352
110722760,4sleep311074-21cpuspeed_turbos21:45:133
103332540,2sleep30-21swapper/300:10:133
210002510,3sleep011-21rcuc/023:20:160
237322480,1sleep10-21swapper/121:00:111
195232480,2sleep00-21swapper/023:15:230
191402480,4sleep011-21rcuc/022:00:260
84632430,2sleep10-21swapper/122:50:251
663399361,11cyclictest7426-21kworker/0:100:09:330
663399342,15cyclictest27958-21kworker/0:322:35:020
663399338,12cyclictest24894-21awk19:50:020
663399331,11cyclictest10388-21kworker/0:021:50:210
663399316,12cyclictest20490-21munin-run19:40:020
663399313,13cyclictest13900-21munin-run19:25:010
663999302,22cyclictest0-21swapper/121:27:111
663999301,23cyclictest0-21swapper/119:33:111
663399308,2cyclictest20889-21kworker/0:021:15:130
663399306,11cyclictest29276-21munin-run20:00:010
663399305,12cyclictest28333-21awk22:25:000
62372300,3sleep19-21rcu_preempt22:45:221
62372300,3sleep19-21rcu_preempt22:45:221
663999292,21cyclictest0-21swapper/121:56:301
663999291,22cyclictest0-21swapper/100:31:231
664699282,6cyclictest0-21swapper/221:55:532
663999281,3cyclictest0-21swapper/122:02:351
663999281,24cyclictest8947-21cat19:15:021
663999281,21cyclictest0-21swapper/120:23:121
663999281,21cyclictest0-21swapper/120:23:111
6639992812,13cyclictest6672-40updatedb00:00:201
6639992812,13cyclictest6672-40updatedb00:00:201
663999280,21cyclictest0-21swapper/120:05:011
663399285,11cyclictest7609-21perf00:05:010
663399285,11cyclictest7609-21perf00:05:010
663399281,10cyclictest22237-21kworker/0:023:34:090
663999279,5cyclictest28871-40dnf23:39:261
663999272,20cyclictest0-21swapper/120:33:361
663999271,4cyclictest14315-40dnf20:37:461
663999271,20cyclictest0-21swapper/120:18:191
663999271,20cyclictest0-21swapper/100:06:281
663999271,17cyclictest4987-21/usr/sbin/munin23:55:261
663399274,20cyclictest0-21swapper/020:20:430
663399274,20cyclictest0-21swapper/020:20:430
663399273,11cyclictest8808-21munin-run19:15:010
663399270,11cyclictest15457-21kworker/0:123:11:410
663999262,19cyclictest0-21swapper/121:36:101
663999261,22cyclictest9111-21diskstats21:40:141
663999261,20cyclictest0-21swapper/123:15:031
663999261,16cyclictest0-21swapper/119:37:341
6633992610,2cyclictest18590-21kworker/0:320:50:570
91462251,3sleep00-21swapper/022:55:020
664699251,14cyclictest0-21swapper/220:54:022
663999252,18cyclictest0-21swapper/123:45:411
6639992513,7cyclictest0-21swapper/123:50:041
6639992513,3cyclictest0-21swapper/120:41:251
663999251,21cyclictest29521-21users21:10:251
663999251,21cyclictest2738-21sort22:40:011
663999251,15cyclictest0-21swapper/123:28:171
663999251,15cyclictest0-21swapper/122:57:591
663999251,15cyclictest0-21swapper/119:41:021
663999250,17cyclictest5649-21sed21:30:231
663399255,4cyclictest30275-21kworker/0:022:29:180
663399251,9cyclictest29665-21kworker/0:121:23:450
663399251,10cyclictest5883-21kworker/0:020:32:460
6633992510,3cyclictest10388-21kworker/0:021:44:130
665099246,10cyclictest16614-21/usr/sbin/munin00:25:133
665099241,8cyclictest0-21swapper/321:31:413
665099240,18cyclictest0-21swapper/323:16:453
6646992414,3cyclictest0-21swapper/221:37:342
6639992413,3cyclictest0-21swapper/120:45:491
6639992413,3cyclictest0-21swapper/120:28:371
6639992412,7cyclictest0-21swapper/121:46:091
663999241,20cyclictest13894-21if_enp1s021:50:161
663999241,15cyclictest0-21swapper/122:42:431
663999241,15cyclictest0-21swapper/120:56:441
663999241,15cyclictest0-21swapper/119:53:201
663999241,15cyclictest0-21swapper/119:47:231
663999241,14cyclictest0-21swapper/100:35:131
663999241,13cyclictest0-21swapper/121:18:331
663999241,11cyclictest27648-21diskstats19:55:131
663999240,19cyclictest0-21swapper/100:23:241
663399249,3cyclictest22569-21/usr/sbin/munin23:25:250
663399249,3cyclictest16319-21/usr/sbin/munin19:30:160
663399246,3cyclictest27958-21kworker/0:323:00:130
663399242,10cyclictest21327-21/usr/sbin/munin20:55:250
663399241,6cyclictest0-21swapper/023:05:160
663399241,4cyclictest0-21swapper/021:11:460
663399241,11cyclictest29164-21/usr/sbin/munin23:40:230
6650992312,3cyclictest87650irq/129-enp1s0-22:00:263
665099231,16cyclictest0-21swapper/323:50:263
665099231,11cyclictest0-21swapper/323:10:183
664699234,10cyclictest27705-21diskmemload21:45:182
664699231,13cyclictest17046-21ntp_states23:10:222
664699231,11cyclictest7646-21uname19:10:172
663999239,2cyclictest10572-21fschecks_time00:10:141
6639992312,3cyclictest0-21swapper/100:27:181
663999231,15cyclictest0-21swapper/123:24:261
663999231,15cyclictest0-21swapper/123:02:551
663999231,15cyclictest0-21swapper/100:17:331
663999231,11cyclictest24988-21expr23:30:121
663999231,11cyclictest1855-21cstates20:10:131
663999231,11cyclictest14808-21netstat23:05:201
663999231,10cyclictest31860-21cpuspeed_turbos20:05:111
663999230,21cyclictest0-21swapper/122:29:371
663999230,21cyclictest0-21swapper/119:25:251
663399237,3cyclictest11399-21kworker/0:119:18:050
663399236,7cyclictest20463-21kworker/0:200:34:540
663399231,4cyclictest0-21swapper/022:39:090
663399231,10cyclictest24150-21/usr/sbin/munin22:15:130
663399231,10cyclictest2161-21/usr/sbin/munin21:25:210
182582231,2sleep20-21swapper/222:00:172
665099229,2cyclictest13764-21gzip20:35:233
665099225,9cyclictest12050irq/123-ahci[0023:45:233
665099221,8cyclictest28637-21threads19:55:233
665099221,6cyclictest0-21swapper/300:00:243
665099221,6cyclictest0-21swapper/300:00:233
665099221,12cyclictest0-21swapper/322:20:503
665099220,6cyclictest0-21swapper/321:29:543
664699221,3cyclictest0-21swapper/223:26:462
664699220,2cyclictest87350irq/126-enp1s0-19:24:412
663999226,6cyclictest0-21swapper/119:20:471
6639992212,2cyclictest0-21swapper/123:11:521
663999221,19cyclictest0-21swapper/122:13:211
663999221,19cyclictest0-21swapper/119:15:131
663999221,18cyclictest24215-21cpuspeed_turbos22:15:131
663999221,14cyclictest0-21swapper/121:24:251
6639992210,3cyclictest0-21swapper/123:40:511
663399229,3cyclictest1176-21kworker/0:220:11:580
663399228,3cyclictest26962-21/usr/sbin/munin23:35:130
663399226,14cyclictest1315-21perf20:10:010
663399221,4cyclictest0-21swapper/000:38:230
663399221,4cyclictest0-21swapper/000:10:380
663399221,2cyclictest808-21snmpd21:45:070
35942221,3sleep03698-21memory22:40:210
187912220,1sleep20-21swapper/200:30:012
116762221,3sleep311854-21grep20:30:243
665099218,3cyclictest0-21swapper/322:30:113
665099216,9cyclictest39-21rcuc/323:31:023
665099215,8cyclictest87650irq/129-enp1s0-22:55:203
665099215,8cyclictest87350irq/126-enp1s0-22:15:143
665099212,10cyclictest27705-21diskmemload23:44:043
665099211,12cyclictest0-21swapper/321:13:063
665099211,11cyclictest26962-21/usr/sbin/munin23:35:253
665099211,11cyclictest22907-21/usr/sbin/munin19:45:133
665099211,11cyclictest11576-21/usr/sbin/munin23:00:133
664699214,7cyclictest0-21swapper/200:05:502
664699213,8cyclictest6284-21diskstats20:20:132
664699213,8cyclictest6284-21diskstats20:20:132
664699212,6cyclictest601250kworker/u8:221:20:112
664699211,7cyclictest27705-21diskmemload00:25:012
664699211,12cyclictest0-21swapper/200:11:042
664699211,11cyclictest1209-21NetworkManager20:09:012
663999211,18cyclictest0-21swapper/122:23:161
663399219,3cyclictest15352-21/usr/sbin/munin21:55:210
663399216,2cyclictest9687-21kworker/0:200:15:570
663399214,4cyclictest13664-21kworker/0:219:29:500
663399214,2cyclictest9687-21kworker/0:200:21:370
663399211,11cyclictest31705-21df23:45:130
663399211,11cyclictest0-21swapper/022:58:020
319902210,3sleep231991-21sendmail_mailst22:30:222
229382210,4sleep2311ktimersoftd/222:10:192
224672211,2sleep222594-21users00:35:282
665099207,3cyclictest87350irq/126-enp1s0-22:05:153
665099205,3cyclictest12050irq/123-ahci[0021:10:003
665099204,10cyclictest39-21rcuc/321:50:223
665099204,10cyclictest0-21swapper/319:35:213
665099202,9cyclictest14124-21/usr/sbin/munin19:25:203
665099201,7cyclictest4371-21users22:40:283
665099201,7cyclictest0-21swapper/323:25:103
665099201,6cyclictest9811-21awk00:10:023
665099201,6cyclictest6906-21df_inode21:35:153
665099201,6cyclictest5937-21/usr/sbin/munin20:20:113
665099201,6cyclictest5937-21/usr/sbin/munin20:20:113
665099201,6cyclictest476-21grep20:05:223
665099201,6cyclictest21773-21ntp_states19:40:203
665099201,6cyclictest13285-21/usr/sbin/munin00:15:223
665099201,6cyclictest0-21swapper/322:29:183
665099201,13cyclictest0-21swapper/321:58:303
665099201,11cyclictest21327-21/usr/sbin/munin20:55:203
665099201,10cyclictest19884-21ntpq00:30:223
665099201,10cyclictest0-21swapper/320:50:133
665099200,5cyclictest0-21swapper/320:45:163
664699206,7cyclictest0-21swapper/221:07:382
664699202,8cyclictest87650irq/129-enp1s0-23:40:452
664699201,8cyclictest0-21swapper/223:30:142
664699201,7cyclictest4061-21switchtime22:40:262
664699201,3cyclictest0-21swapper/219:19:502
664699201,2cyclictest0-21swapper/222:51:382
664699201,12cyclictest0-21swapper/222:22:582
664699201,10cyclictest14735-21/usr/sbin/munin20:40:232
664699201,10cyclictest0-21swapper/200:32:012
664699200,3cyclictest87550irq/128-enp1s0-19:44:532
664699200,3cyclictest0-21swapper/220:46:062
664699200,2cyclictest87450irq/127-enp1s0-20:12:412
663999205,6cyclictest30743-21/usr/sbin/munin22:30:201
663999203,14cyclictest198702chrt20:50:171
663999200,18cyclictest0-21swapper/121:05:261
663399207,4cyclictest20889-21kworker/0:021:02:450
663399206,5cyclictest0-21swapper/023:52:390
663399205,4cyclictest6283-21kworker/0:121:37:090
663399201,5cyclictest0-21swapper/021:33:140
663399201,4cyclictest19366-21kworker/0:222:07:090
55102200,2sleep30-21swapper/320:17:363
665099198,3cyclictest12837-21memory19:20:183
665099198,3cyclictest0-21swapper/321:40:253
665099196,4cyclictest15040-21sensors_temp23:05:233
665099195,3cyclictest549-21/usr/sbin/munin22:35:133
665099195,3cyclictest12050irq/123-ahci[0021:05:023
665099194,8cyclictest87350irq/126-enp1s0-22:45:193
665099194,8cyclictest87350irq/126-enp1s0-22:45:193
665099193,4cyclictest87350irq/126-enp1s0-21:20:003
665099191,6cyclictest7529-21diskstats22:50:123
665099191,6cyclictest3982-21fw_packets23:55:153
665099191,6cyclictest3513-21latency_hist20:15:013
665099191,5cyclictest8954-21irqstats20:25:163
6650991915,3cyclictest0-21swapper/323:20:183
665099191,11cyclictest0-21swapper/321:23:493
665099191,10cyclictest22891-21sed22:10:203
665099191,10cyclictest21009-21/usr/sbin/munin00:35:153
665099190,7cyclictest10486-21if_enp2s019:15:173
664699193,8cyclictest1327-21cpuspeed_turbos23:50:132
664699193,4cyclictest18648-21fschecks_time23:15:142
664699193,3cyclictest0-21swapper/221:44:062
664699193,2cyclictest87550irq/128-enp1s0-20:58:422
664699191,9cyclictest13472-21kworker/u8:223:35:172
664699191,9cyclictest0-21swapper/223:57:332
664699191,9cyclictest0-21swapper/222:06:522
664699191,9cyclictest0-21swapper/221:27:152
664699191,7cyclictest0-21swapper/222:17:262
664699191,7cyclictest0-21swapper/219:26:292
664699191,6cyclictest27705-21diskmemload00:00:152
664699191,6cyclictest27705-21diskmemload00:00:152
664699191,6cyclictest13785-21/usr/sbin/munin23:05:112
664699191,4cyclictest0-21swapper/223:46:102
664699191,10cyclictest0-21swapper/221:13:412
663399197,3cyclictest21591-21kworker/0:122:13:060
663399195,6cyclictest0-21swapper/022:48:130
663399195,6cyclictest0-21swapper/022:48:130
663399194,3cyclictest18590-21kworker/0:320:48:050
663399193,4cyclictest20279-21kworker/0:319:42:140
663399191,9cyclictest14071-21kworker/0:000:26:090
6633991912,4cyclictest827-21runrttasks20:35:230
663399191,10cyclictest25103-21/usr/sbin/munin19:50:190
6633991910,3cyclictest14496-21kworker/0:120:43:330
150602191,3sleep20-21swapper/221:54:192
665099181,9cyclictest14735-21/usr/sbin/munin20:40:213
665099181,6cyclictest0-21swapper/319:50:183
665099181,6cyclictest0-21swapper/300:20:153
665099181,10cyclictest6012-21kworker/u8:219:34:383
664699185,3cyclictest87650irq/129-enp1s0-23:00:122
664699185,3cyclictest87650irq/129-enp1s0-20:25:172
664699184,7cyclictest0-21swapper/219:59:092
664699183,9cyclictest0-21swapper/222:26:332
664699181,9cyclictest22907-21/usr/sbin/munin19:45:172
664699181,9cyclictest0-21swapper/222:59:172
664699181,5cyclictest808-21snmpd19:39:422
664699181,5cyclictest3742-21/usr/sbin/munin20:15:142
664699181,5cyclictest20595-21cat23:20:142
6646991812,4cyclictest24783-21smart_sda21:00:232
663399185,4cyclictest29008-21kworker/0:020:02:300
663399181,9cyclictest0-21swapper/023:55:490
663399181,5cyclictest25731-21/usr/sbin/munin21:05:160
22272181,2sleep20-21swapper/222:36:012
665099171,6cyclictest0-21swapper/320:00:173
664699171,7cyclictest311622sleep220:01:012
6646991712,3cyclictest31236-21ls21:15:222
664699170,4cyclictest0-21swapper/220:31:452
664699170,2cyclictest87450irq/127-enp1s0-20:35:052
663399174,4cyclictest1176-21kworker/0:220:15:060
6633991711,3cyclictest8419-21expr20:25:120
664699162,5cyclictest12894-21hddtemp_smartct00:15:162
6646991611,3cyclictest16396-21cat19:30:102
664699152,7cyclictest0-21swapper/219:54:122
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional