You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-09 - 23:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot3s.osadl.org (updated Tue Jun 09, 2026 12:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
49872234163,58sleep20-21swapper/207:06:242
51992203166,24sleep10-21swapper/107:09:091
49382202166,24sleep30-21swapper/307:05:463
50622200163,25sleep00-21swapper/007:07:230
2017621700,4sleep3536999cyclictest11:20:263
1755521530,4sleep1535899cyclictest12:30:111
226621450,3sleep0535399cyclictest08:15:020
53659911810,106cyclictest0-21swapper/212:35:572
5365999289,2cyclictest0-21swapper/212:16:112
5822690,2sleep10-21swapper/109:25:011
295402460,5sleep029543-21idleruntime09:15:170
295402460,5sleep029543-21idleruntime09:15:160
224242460,2sleep10-21swapper/110:12:291
96012420,1sleep30-21swapper/309:45:023
535399366,14cyclictest25456-21basename11:35:000
536999341,24cyclictest0-21swapper/311:48:303
536999331,18cyclictest0-21swapper/309:28:023
535399324,13cyclictest9451-21munin-run09:45:020
535399316,12cyclictest20438-21munin-run10:10:010
536999300,23cyclictest0-21swapper/311:38:303
5353993016,8cyclictest14630-21cut08:40:220
536999280,2cyclictest0-21swapper/312:02:223
535899281,5cyclictest0-21swapper/112:36:181
536999272,9cyclictest29552-40dnf11:41:323
536999271,12cyclictest0-21swapper/311:05:293
536999271,11cyclictest0-21swapper/307:11:113
536999270,11cyclictest0-21swapper/309:57:413
536999270,11cyclictest0-21swapper/309:39:543
5365992712,2cyclictest0-21swapper/209:40:552
535899271,23cyclictest15478-21timerwakeupswit09:55:251
535899271,16cyclictest0-21swapper/108:45:141
535399279,15cyclictest31433-21idleruntime-cro10:35:010
535399273,11cyclictest14453-21idleruntime-cro11:10:010
536999261,8cyclictest0-21swapper/308:20:173
536999261,12cyclictest0-21swapper/311:15:173
536999261,12cyclictest0-21swapper/310:47:093
536999261,11cyclictest0-21swapper/308:43:273
536999261,10cyclictest0-21swapper/307:26:323
536999260,9cyclictest0-21swapper/307:31:383
536999260,11cyclictest0-21swapper/311:29:023
535899261,12cyclictest0-21swapper/108:40:241
535899261,10cyclictest0-21swapper/109:36:291
5358992610,3cyclictest15510-21uniq07:30:141
535399261,12cyclictest4430-21munin-run08:20:020
535399260,12cyclictest0-21swapper/009:50:020
536999251,9cyclictest0-21swapper/307:44:083
536999251,11cyclictest0-21swapper/307:36:363
536999251,10cyclictest0-21swapper/307:20:563
536999250,9cyclictest0-21swapper/311:01:253
536999250,12cyclictest0-21swapper/309:50:493
536999250,11cyclictest0-21swapper/312:08:303
536999250,10cyclictest0-21swapper/312:23:333
535899257,7cyclictest827-21runrttasks09:14:371
535899251,21cyclictest17730-21latency11:15:191
535899251,16cyclictest0-21swapper/108:15:561
535899251,15cyclictest13357-21unixbench_multi09:50:251
535899251,15cyclictest0-21swapper/108:57:411
535899251,15cyclictest0-21swapper/108:31:401
535399258,14cyclictest17284-21idleruntime-cro12:30:010
260302250,4sleep232-21ksoftirqd/207:55:122
10952250,1sleep10-21swapper/109:25:131
536999248,3cyclictest0-21swapper/308:04:553
536999248,3cyclictest0-21swapper/308:04:553
536999247,3cyclictest0-21swapper/308:51:143
536999241,11cyclictest0-21swapper/312:30:493
536999241,11cyclictest0-21swapper/311:53:223
536999241,10cyclictest0-21swapper/308:29:033
536999240,8cyclictest0-21swapper/310:51:413
536999240,7cyclictest0-21swapper/308:55:063
536999240,19cyclictest0-21swapper/312:28:413
536999240,10cyclictest0-21swapper/310:33:333
536999240,10cyclictest0-21swapper/309:23:223
535899245,8cyclictest827-21runrttasks10:31:321
535899244,10cyclictest5608-21switchtime12:00:251
5358992420,3cyclictest0-21swapper/109:34:361
5358992414,2cyclictest0-21swapper/108:06:351
5358992414,1cyclictest0-21swapper/111:25:231
535899241,20cyclictest6917-21grep08:25:121
535899241,20cyclictest0-21swapper/112:15:121
535899241,15cyclictest0-21swapper/109:09:351
535899241,15cyclictest0-21swapper/108:10:131
535899241,15cyclictest0-21swapper/107:40:081
535899241,11cyclictest8795-21diskstats07:15:141
535899241,11cyclictest32603-21df_inode11:50:131
535899241,11cyclictest3076-21users10:40:281
535899241,11cyclictest19562-21/usr/sbin/munin10:05:251
535899241,11cyclictest11195-21perl07:20:151
535899241,10cyclictest24156-21hwlatdetect11:30:161
535399248,13cyclictest2863-21idleruntime-cro09:30:020
535399244,10cyclictest17708-21kernelversion11:15:190
536999239,3cyclictest0-21swapper/307:55:203
536999235,9cyclictest0-21swapper/308:45:063
536999231,9cyclictest0-21swapper/310:40:133
536999231,9cyclictest0-21swapper/308:15:073
536999231,8cyclictest0-21swapper/307:52:163
536999231,7cyclictest0-21swapper/308:05:593
536999231,11cyclictest0-21swapper/312:36:573
536999231,11cyclictest0-21swapper/308:37:333
536999231,10cyclictest0-21swapper/309:30:033
536999230,8cyclictest0-21swapper/310:00:213
536599231,6cyclictest0-21swapper/211:41:302
536599231,4cyclictest0-21swapper/212:20:212
535899239,11cyclictest10013-21switchtime12:10:251
535899231,4cyclictest10348-21hddtemp_smartct09:45:161
5358992312,3cyclictest0-21swapper/108:51:161
5358992312,2cyclictest0-21swapper/107:52:271
535899231,19cyclictest0-21swapper/110:25:121
535899231,19cyclictest0-21swapper/110:25:121
535899231,15cyclictest0-21swapper/110:02:021
535899231,14cyclictest0-21swapper/110:24:371
535899231,14cyclictest0-21swapper/107:39:421
535899231,14cyclictest0-21swapper/107:27:031
535899231,10cyclictest8615-21cat12:10:011
535899230,14cyclictest0-21swapper/110:15:571
536999228,3cyclictest0-21swapper/310:06:233
536999228,3cyclictest0-21swapper/308:11:453
536999223,3cyclictest0-21swapper/309:03:223
536999221,9cyclictest0-21swapper/310:37:273
536999221,9cyclictest0-21swapper/310:25:353
536999221,9cyclictest0-21swapper/310:25:353
536999221,7cyclictest0-21swapper/310:17:273
536999221,14cyclictest0-21swapper/312:12:363
536999221,13cyclictest0-21swapper/309:05:463
536999220,9cyclictest0-21swapper/311:30:143
536999220,9cyclictest0-21swapper/309:14:183
536999220,11cyclictest0-21swapper/312:19:133
5365992219,2cyclictest0-21swapper/210:30:532
536599221,6cyclictest0-21swapper/210:01:382
536599221,11cyclictest2141-21smart_sda09:25:242
535899222,2cyclictest87650irq/129-enp1s0-12:25:211
535899221,9cyclictest0-21swapper/111:44:131
535899221,3cyclictest6616-21ntp_states07:10:201
535899221,19cyclictest0-21swapper/112:20:141
535899221,19cyclictest0-21swapper/110:52:441
535899221,15cyclictest0-21swapper/111:49:191
535899221,11cyclictest0-21swapper/108:35:181
535899221,10cyclictest20145-21ntpq11:20:241
535899220,15cyclictest0-21swapper/107:55:331
535899220,14cyclictest0-21swapper/109:01:351
535899220,14cyclictest0-21swapper/108:20:051
535899220,14cyclictest0-21swapper/107:45:301
535399221,4cyclictest21065-21awk11:25:000
535399221,10cyclictest26414-21diskmemload10:15:010
535399221,10cyclictest19832-21idleruntime-cro08:55:000
535399220,11cyclictest0-21swapper/011:40:310
536999217,4cyclictest15664-21/usr/sbin/munin11:10:183
536999211,6cyclictest0-21swapper/310:23:073
536999211,14cyclictest0-21swapper/307:18:373
536999210,9cyclictest0-21swapper/310:14:073
536999210,12cyclictest0-21swapper/309:18:223
536999210,12cyclictest0-21swapper/309:18:223
536999210,12cyclictest0-21swapper/307:46:543
536599213,6cyclictest32-21ksoftirqd/210:47:232
536599211,6cyclictest29227-21perf10:30:012
536599211,6cyclictest29227-21perf10:30:002
536599211,13cyclictest0-21swapper/208:15:202
536599211,11cyclictest15648-21/usr/sbin/munin08:45:132
536599211,11cyclictest12492-21/usr/sbin/munin11:05:242
535899218,2cyclictest31685-21cpu10:35:151
5358992118,2cyclictest0-21swapper/111:00:181
5358992112,6cyclictest19464-21kworker/u8:008:00:251
5358992112,6cyclictest19464-21kworker/u8:008:00:251
535899211,18cyclictest23-21ksoftirqd/111:10:321
535899211,10cyclictest0-21swapper/111:38:451
535399214,9cyclictest0-21swapper/009:39:290
535399213,10cyclictest0-21swapper/007:10:240
535399211,5cyclictest0-21swapper/012:20:410
535399210,10cyclictest0-21swapper/011:37:540
536999201,4cyclictest0-21swapper/311:55:343
536999200,10cyclictest0-21swapper/309:49:573
536599207,3cyclictest0-21swapper/211:29:422
536599204,5cyclictest26414-21diskmemload11:00:212
536599202,5cyclictest7251-21munin-run09:40:012
536599202,3cyclictest2709-21gltestperf11:55:182
536599201,7cyclictest21598-21cat07:45:012
536599201,6cyclictest8849-21if_enp2s010:55:172
536599201,6cyclictest18788-21sed07:35:242
536599201,6cyclictest18474-21users11:15:282
5365992014,3cyclictest29394-21processes08:00:212
5365992014,3cyclictest29394-21processes08:00:212
536599201,3cyclictest0-21swapper/211:20:022
536599201,14cyclictest0-21swapper/210:14:132
536599201,12cyclictest0-21swapper/210:39:212
536599201,12cyclictest0-21swapper/209:05:172
536599201,10cyclictest5553-21/usr/sbin/munin07:10:132
536599201,10cyclictest26414-21diskmemload12:25:282
536599201,10cyclictest20042-21/usr/sbin/munin08:55:222
536599201,10cyclictest15751-21ntp_states11:10:212
535899206,3cyclictest9080-21sshd09:40:281
535899201,9cyclictest0-21swapper/111:56:571
535899201,2cyclictest0-21swapper/110:56:231
535899201,2cyclictest0-21swapper/110:47:421
535899201,18cyclictest0-21swapper/111:07:211
535399205,5cyclictest0-21swapper/012:33:060
535399201,4cyclictest0-21swapper/010:17:540
535399201,10cyclictest32034-21fschecks_count10:35:160
535399200,11cyclictest9-21rcu_preempt09:30:240
535399200,10cyclictest0-21swapper/011:13:290
536999191,10cyclictest8087-21/usr/sbin/munin10:55:183
536599194,4cyclictest8564-1kworker/u9:111:46:022
536599194,3cyclictest23226-21needreboot09:00:192
536599193,4cyclictest5937-21cpu10:50:132
536599191,7cyclictest18646-21proc_pri12:30:202
536599191,7cyclictest0-21swapper/209:32:462
536599191,6cyclictest7916-21ntpq08:25:202
536599191,6cyclictest5880-21rt-features08:20:222
536599191,6cyclictest18825-21needreboot08:50:202
536599191,6cyclictest13921-21open_files07:25:212
536599191,6cyclictest0-21swapper/212:10:262
536599191,6cyclictest0-21swapper/210:22:372
536599191,6cyclictest0-21swapper/208:10:162
536599191,5cyclictest10642-21/usr/sbin/munin07:20:192
536599191,5cyclictest0-21swapper/209:45:152
536599191,4cyclictest0-21swapper/209:24:422
536599191,12cyclictest0-21swapper/212:05:122
536599191,12cyclictest0-21swapper/208:35:172
536599191,11cyclictest0-21swapper/209:10:122
536599191,10cyclictest32261-21/usr/sbin/munin11:50:142
535899190,18cyclictest0-21swapper/109:17:201
535899190,18cyclictest0-21swapper/109:17:201
535399199,7cyclictest10662-21sshd08:30:290
535399192,3cyclictest15129-21ntp_states09:55:220
535399191,9cyclictest0-21swapper/012:37:310
535399191,6cyclictest0-21swapper/011:51:140
5353991914,3cyclictest15403-21grep07:30:140
5353991913,4cyclictest31652-21grep09:20:160
5353991913,3cyclictest9510-21ntp_states07:15:210
535399191,2cyclictest13797-21chrt09:54:380
535399191,10cyclictest27599-21diskstats10:25:130
535399191,10cyclictest27599-21diskstats10:25:130
535399190,8cyclictest0-21swapper/011:02:050
53462191,14sleep25374-21ntp_states12:00:222
536999184,8cyclictest0-21swapper/308:30:253
536599184,8cyclictest6351-21kworker/u8:109:56:032
536599184,4cyclictest24585-21if_enp2s007:50:172
536599181,9cyclictest14439-21needreboot08:40:202
536599181,8cyclictest0-21swapper/207:30:122
536599181,7cyclictest0-21swapper/210:05:542
536599181,6cyclictest0-21swapper/210:15:292
536599181,5cyclictest13002-21sed09:50:212
536599181,4cyclictest0-21swapper/211:35:172
5365991813,3cyclictest30089-21sendmail_mailtr09:15:232
5365991813,3cyclictest30089-21sendmail_mailtr09:15:232
535399183,3cyclictest19464-21kworker/u8:009:11:040
535399182,7cyclictest5888-21/usr/sbin/munin10:50:170
535399182,10cyclictest0-21swapper/011:58:450
535399181,7cyclictest808-21snmpd10:44:280
535399181,3cyclictest20977-21sed08:55:190
535399181,3cyclictest10297-21users12:10:280
535399181,15cyclictest24232-21awk09:05:000
535399181,11cyclictest0-21swapper/008:25:100
535399181,10cyclictest0-21swapper/012:03:350
535399181,10cyclictest0-21swapper/010:20:170
535399180,8cyclictest0-21swapper/011:47:130
252492181,3sleep20-21swapper/211:32:422
536599172,3cyclictest22623-21needreboot07:45:192
536599171,8cyclictest12050irq/123-ahci[0007:20:022
536599171,6cyclictest0-21swapper/210:40:532
536599171,6cyclictest0-21swapper/208:05:482
535399178,3cyclictest0-21swapper/010:45:580
535399178,2cyclictest0-21swapper/012:05:170
535399176,3cyclictest17850-21users10:00:270
535399176,3cyclictest165050irq/12-ttyS008:35:200
535399171,7cyclictest0-21swapper/007:39:320
535399171,4cyclictest0-21swapper/008:45:150
5353991713,3cyclictest0-21swapper/012:15:530
535399171,14cyclictest7474-21kworker/0:107:25:330
5353991711,4cyclictest11862-21gunzip07:20:220
535399171,10cyclictest0-21swapper/011:28:430
5353991710,4cyclictest8725-21sshd10:55:150
536599164,3cyclictest0-21swapper/208:30:082
5353991612,3cyclictest0-21swapper/008:24:340
535399161,14cyclictest0-21swapper/007:56:370
5353991611,3cyclictest21084-21seq07:41:060
5353991610,3cyclictest24445-21/usr/sbin/munin09:05:180
5353991610,3cyclictest24104-21expr07:50:140
319182160,2sleep00-21swapper/008:05:280
535399151,5cyclictest29278-21ntp_states08:00:210
535399151,5cyclictest29278-21ntp_states08:00:210
535399151,5cyclictest0-21swapper/007:45:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional