You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-01 - 11:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Tue Sep 01, 2026 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
70752240166,24sleep20-21swapper/219:07:492
70072235164,58sleep10-21swapper/119:06:551
68962202167,23sleep00-21swapper/019:05:300
52842202166,24sleep30-21swapper/319:05:083
1296921320,3sleep3734699cyclictest19:20:143
7341996865,2cyclictest0-21swapper/223:51:152
7341996460,3cyclictest0-21swapper/222:58:572
313922600,2sleep1221ktimersoftd/123:40:261
7341995753,3cyclictest0-21swapper/200:18:162
20942570,4sleep00-21swapper/022:35:190
20942570,4sleep00-21swapper/022:35:190
116272570,4sleep00-21swapper/000:15:110
236122460,1sleep20-21swapper/222:10:202
134032460,3sleep20-21swapper/219:20:172
734699351,6cyclictest20160-21df20:50:133
733599351,12cyclictest20479-21diskstats00:35:151
734699345,4cyclictest29030-21awk22:25:013
733599339,3cyclictest28397-21diskmemload00:26:211
733599331,11cyclictest21718-21sh22:05:251
734699321,14cyclictest27997-21df23:35:133
732999329,11cyclictest0-21swapper/020:20:020
732999329,11cyclictest0-21swapper/020:20:020
732999326,12cyclictest11121-21munin-run00:15:010
734699311,14cyclictest0-21swapper/300:20:123
733599317,11cyclictest14730-21expr23:05:141
732999317,11cyclictest24634-21idleruntime-cro22:15:000
734699301,13cyclictest30878-21hddtemp_smartct20:00:173
732999305,12cyclictest10816-21munin-run20:30:020
732999301,14cyclictest0-21swapper/022:20:000
732999301,13cyclictest7652-21munin-run22:50:020
734699291,12cyclictest0-21swapper/322:57:383
732999293,12cyclictest8930-21perf00:10:010
89942280,4sleep230-21rcuc/222:50:212
732999282,12cyclictest17420-21perf20:45:010
734699273,7cyclictest0-21swapper/323:17:313
734699271,12cyclictest0-21swapper/323:57:413
734699270,14cyclictest0-21swapper/321:02:433
733599271,11cyclictest0-21swapper/120:10:211
733599270,6cyclictest0-21swapper/121:27:161
734699265,3cyclictest0-21swapper/320:47:393
734699261,3cyclictest87350irq/126-enp1s0-19:35:163
7346992612,1cyclictest0-21swapper/322:45:153
734699261,15cyclictest14931-21cpuspeed_turbos19:25:143
734699261,12cyclictest0-21swapper/321:30:293
734699261,12cyclictest0-21swapper/319:50:133
734699261,12cyclictest0-21swapper/319:49:143
734699261,11cyclictest0-21swapper/321:18:573
734699261,11cyclictest0-21swapper/321:10:493
734199261,12cyclictest0-21swapper/219:47:332
733599261,4cyclictest5766-21tr22:45:121
7335992614,2cyclictest28397-21diskmemload00:01:351
733599261,3cyclictest17400-21timerandwakeup21:55:261
733599261,16cyclictest0-21swapper/123:19:111
733599260,16cyclictest0-21swapper/121:10:151
273122260,3sleep127317-21unixbench_multi19:50:251
734699250,12cyclictest0-21swapper/322:12:023
734699250,12cyclictest0-21swapper/322:07:503
733599252,10cyclictest0-21swapper/120:20:011
733599252,10cyclictest0-21swapper/120:20:011
733599251,15cyclictest0-21swapper/123:55:231
733599251,15cyclictest0-21swapper/122:04:061
733599251,11cyclictest2751-21cat21:25:021
733599250,16cyclictest0-21swapper/121:15:361
732999251,13cyclictest8302-21awk00:05:010
734699241,12cyclictest0-21swapper/323:49:393
734699241,12cyclictest0-21swapper/323:49:393
734699241,11cyclictest0-21swapper/320:42:213
733599249,2cyclictest12578-21sort21:45:181
733599247,3cyclictest0-21swapper/122:50:191
733599246,9cyclictest0-21swapper/121:40:191
733599241,5cyclictest0-21swapper/100:12:571
733599241,3cyclictest909-21sssd_nss20:50:161
733599241,3cyclictest30624-21uniq20:00:151
7335992413,3cyclictest0-21swapper/119:43:531
733599241,2cyclictest0-21swapper/122:59:011
733599241,15cyclictest0-21swapper/123:46:441
733599241,15cyclictest0-21swapper/123:46:441
733599241,15cyclictest0-21swapper/119:23:481
733599241,14cyclictest0-21swapper/100:20:571
733599241,11cyclictest28353-21diskstats19:55:141
733599241,11cyclictest18284-21diskstats00:30:151
733599241,11cyclictest12616-21df_inode23:00:141
733599241,10cyclictest11402-21awk00:15:111
733599240,16cyclictest0-21swapper/119:37:321
732999241,11cyclictest23037-21idleruntime-cro23:25:000
165052240,3sleep3734699cyclictest00:25:193
734699236,3cyclictest31711-21runrttasks23:43:353
734699231,11cyclictest0-21swapper/321:57:063
734199237,6cyclictest0-21swapper/221:10:122
7335992312,3cyclictest0-21swapper/123:27:361
7335992312,3cyclictest0-21swapper/122:25:181
733599231,15cyclictest0-21swapper/120:55:221
733599231,10cyclictest24281-21cpu21:00:131
733599230,5cyclictest0-21swapper/123:10:191
733599230,15cyclictest0-21swapper/122:38:161
733599230,15cyclictest0-21swapper/122:38:161
733599230,14cyclictest0-21swapper/123:36:051
732999231,10cyclictest9860-21perf22:55:020
734699221,11cyclictest3806-21df22:40:123
734699220,14cyclictest0-21swapper/323:01:013
733599227,3cyclictest0-21swapper/121:54:131
733599224,3cyclictest0-21swapper/119:11:201
733599222,10cyclictest28403-21munin-run21:10:011
7335992213,2cyclictest0-21swapper/121:36:521
7335992212,2cyclictest0-21swapper/122:21:241
733599221,15cyclictest0-21swapper/120:08:101
733599221,10cyclictest15189-21cat20:39:571
733599221,10cyclictest0-21swapper/120:47:141
734699215,4cyclictest87350irq/126-enp1s0-20:15:223
734699215,4cyclictest87350irq/126-enp1s0-20:15:223
734699212,2cyclictest87350irq/126-enp1s0-23:07:543
734699211,12cyclictest0-21swapper/319:10:213
734699211,11cyclictest2580-21df20:10:143
734199211,6cyclictest0-21swapper/222:20:172
734199211,6cyclictest0-21swapper/221:24:322
734199210,5cyclictest8564-1kworker/u9:121:48:292
733599217,3cyclictest0-21swapper/120:34:381
733599211,9cyclictest1103-21cat22:35:011
7335992115,3cyclictest10827-21fschecks_count19:15:161
733599211,15cyclictest0-21swapper/121:34:121
733599211,14cyclictest0-21swapper/120:42:191
733599211,11cyclictest17353-21/usr/sbin/munin19:30:121
733599211,11cyclictest0-21swapper/119:45:241
732999216,5cyclictest0-21swapper/022:30:090
732999214,7cyclictest2907-21kworker/u8:023:07:060
732999212,16cyclictest4215-21munin-run20:15:010
732999211,9cyclictest11-21rcuc/022:05:010
732999211,7cyclictest12343-21sed20:30:240
732999211,12cyclictest0-21swapper/022:22:240
304642210,2sleep30-21swapper/322:25:233
734699209,3cyclictest28006-21/usr/sbin/munin19:55:203
734699209,3cyclictest26721-21chrt23:30:233
734699204,7cyclictest28397-21diskmemload21:54:523
734699203,10cyclictest0-21swapper/322:16:313
734699202,5cyclictest28018-21users21:05:263
734699202,10cyclictest8842-21/usr/sbin/munin20:25:183
734699202,10cyclictest6637-21/usr/sbin/munin20:20:283
734699202,10cyclictest11651-21/usr/sbin/munin21:45:223
734699201,3cyclictest0-21swapper/319:30:253
734699201,2cyclictest0-21swapper/300:00:133
734699201,15cyclictest0-21swapper/322:50:093
734699201,12cyclictest0-21swapper/322:38:423
734699201,12cyclictest0-21swapper/322:38:423
734699201,12cyclictest0-21swapper/320:34:293
734199204,7cyclictest31752-21missed_timers21:15:202
734199202,5cyclictest0-21swapper/223:30:192
734199201,5cyclictest6088-21kworker/u8:100:25:042
733599201,9cyclictest0-21swapper/123:20:581
733599201,14cyclictest0-21swapper/123:53:141
733599201,14cyclictest0-21swapper/122:16:291
733599201,10cyclictest7749-21open_inodes20:20:231
733599201,10cyclictest22649-21/usr/sbin/munin22:10:231
733599200,15cyclictest0-21swapper/100:05:501
732999208,11cyclictest0-21swapper/022:00:010
732999202,10cyclictest19216-21/usr/sbin/munin19:35:260
7329992014,4cyclictest5724-21cut21:30:160
732999201,13cyclictest0-21swapper/021:00:190
732999201,11cyclictest11366-21sshd21:44:230
734699198,3cyclictest13238-21/usr/sbin/munin20:35:233
734699196,8cyclictest87350irq/126-enp1s0-00:35:143
734699194,9cyclictest0-21swapper/300:12:423
734699194,8cyclictest87350irq/126-enp1s0-21:35:103
734699194,4cyclictest21238-21cat23:20:113
734699192,4cyclictest0-21swapper/323:25:163
734699192,11cyclictest0-21swapper/323:13:533
734699191,5cyclictest12148-21irqstats00:15:173
734699191,14cyclictest0-21swapper/320:55:253
734699191,14cyclictest0-21swapper/300:07:503
734699191,10cyclictest0-21swapper/323:51:233
734699191,10cyclictest0-21swapper/322:30:403
734699190,7cyclictest18021-21cpuspeed_turbos00:30:143
734199197,3cyclictest0-21swapper/220:22:032
734199196,4cyclictest157832switchtime23:05:242
734199194,6cyclictest30-21rcuc/219:30:222
734199191,9cyclictest0-21swapper/222:05:262
734199191,5cyclictest0-21swapper/221:52:002
7341991913,4cyclictest9490-21head20:25:152
733599196,3cyclictest0-21swapper/120:25:361
733599191,9cyclictest15426-21tail19:25:161
7335991911,3cyclictest0-21swapper/123:31:531
733599190,17cyclictest0-21swapper/122:40:181
732999197,9cyclictest1246-21sshd20:05:240
732999195,5cyclictest0-21swapper/000:35:160
732999191,8cyclictest0-21swapper/021:35:570
732999191,6cyclictest0-21swapper/023:36:010
732999191,6cyclictest0-21swapper/000:23:420
732999191,5cyclictest0-21swapper/022:59:410
732999191,3cyclictest0-21swapper/000:25:340
732999191,12cyclictest0-21swapper/023:00:140
732999191,12cyclictest0-21swapper/022:26:310
732999191,10cyclictest32343-21cat23:45:120
732999191,10cyclictest32343-21cat23:45:120
732999191,10cyclictest29861-21/usr/sbin/munin23:40:240
38602190,4sleep0732999cyclictest21:25:190
734699183,9cyclictest0-21swapper/321:23:063
734699181,13cyclictest0-21swapper/321:40:183
734699181,12cyclictest0-21swapper/321:28:413
734699181,11cyclictest0-21swapper/322:01:203
734199185,5cyclictest30-21rcuc/222:35:382
734199185,5cyclictest30-21rcuc/222:35:382
734199184,4cyclictest14998-21timerwakeupswit00:20:252
734199182,5cyclictest17562-21sed21:55:282
734199181,8cyclictest18631-21netstat20:45:202
734199181,6cyclictest0-21swapper/220:16:232
734199181,6cyclictest0-21swapper/220:16:232
734199181,6cyclictest0-21swapper/200:35:352
734199181,6cyclictest0-21swapper/200:09:452
734199181,5cyclictest0-21swapper/223:40:212
734199181,5cyclictest0-21swapper/222:01:102
734199181,15cyclictest31271-21ntp_states20:00:232
734199181,15cyclictest0-21swapper/223:18:242
734199181,15cyclictest0-21swapper/220:05:242
734199181,14cyclictest3779-21unixbench_singl20:10:262
734199180,5cyclictest0-21swapper/200:34:202
732999186,11cyclictest7-21ksoftirqd/019:25:220
732999183,2cyclictest87350irq/126-enp1s0-00:31:060
732999182,8cyclictest0-21swapper/023:55:130
732999182,14cyclictest0-21swapper/019:46:570
732999181,9cyclictest0-21swapper/023:31:560
732999181,9cyclictest0-21swapper/023:10:070
732999181,3cyclictest0-21swapper/021:48:120
732999181,11cyclictest0-21swapper/020:45:330
732999181,10cyclictest22024-21/usr/sbin/munin20:55:230
732999181,10cyclictest0-21swapper/021:53:180
732999181,10cyclictest0-21swapper/020:03:390
732999180,5cyclictest0-21swapper/023:15:260
6512181,2sleep20-21swapper/222:31:122
308902181,3sleep20-21swapper/222:26:022
276802181,2sleep00-21swapper/019:53:550
267532181,2sleep20-21swapper/222:19:202
734699174,5cyclictest755-21rngd19:40:043
734699173,7cyclictest0-21swapper/320:07:233
734699171,4cyclictest11520-40dnf19:15:233
734199176,4cyclictest0-21swapper/200:11:332
734199172,7cyclictest32-21ksoftirqd/219:25:202
734199172,6cyclictest26614-21irqstats19:50:172
734199172,6cyclictest15819-21fschecks_count20:40:152
734199171,8cyclictest26923-21kworker/2:220:55:292
734199171,7cyclictest7630-21fschecks_count21:35:142
734199171,7cyclictest0-21swapper/223:47:592
734199171,7cyclictest0-21swapper/223:47:592
734199171,7cyclictest0-21swapper/222:48:022
734199171,7cyclictest0-21swapper/221:42:492
734199171,6cyclictest19936-21cpuspeed_turbos20:50:122
734199171,6cyclictest0-21swapper/222:44:082
734199171,6cyclictest0-21swapper/221:31:552
734199171,6cyclictest0-21swapper/219:43:562
734199171,4cyclictest0-21swapper/223:35:552
7341991712,3cyclictest2861-21acpi21:25:132
7341991712,3cyclictest21178-21idleruntime-cro19:40:012
734199171,14cyclictest28397-21diskmemload23:02:262
7341991711,4cyclictest27951-21grep21:05:262
7341991711,4cyclictest0-21swapper/219:15:232
734199171,10cyclictest0-21swapper/223:14:002
734199170,15cyclictest0-21swapper/220:38:502
732999178,3cyclictest0-21swapper/022:41:050
732999176,3cyclictest87550irq/128-enp1s0-22:05:160
732999173,6cyclictest0-21swapper/021:17:560
732999171,8cyclictest3755-21seq23:53:400
732999171,6cyclictest0-21swapper/021:10:220
732999171,6cyclictest0-21swapper/019:20:120
732999171,6cyclictest0-21swapper/019:19:270
732999171,5cyclictest0-21swapper/019:55:280
732999171,4cyclictest1338-21ls21:20:160
732999171,14cyclictest0-21swapper/021:05:020
7329991711,3cyclictest7810-21ls19:10:140
7329991710,5cyclictest0-21swapper/023:25:200
244722170,8sleep230-21rcuc/221:00:142
151182170,2sleep00-21swapper/020:39:010
82892160,2sleep00-21swapper/020:21:000
734199161,6cyclictest9123-21users19:10:272
7341991612,3cyclictest0-21swapper/223:58:122
7341991612,3cyclictest0-21swapper/220:30:512
734199161,14cyclictest0-21swapper/219:58:272
7341991611,3cyclictest24765-21unixbench_multi23:25:252
7341991610,3cyclictest21070-21/usr/sbin/munin23:20:232
7329991611,3cyclictest21989-21perl19:40:160
7329991611,3cyclictest20514-21hwlatdetect20:50:160
734199151,8cyclictest0-21swapper/200:00:142
732999158,4cyclictest17180-21cpuspeed_turbos19:30:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional