You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-15 - 21:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Sun Mar 15, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1257029582,9sleep00-21swapper/007:06:010
1275828976,9sleep20-21swapper/207:08:282
1279428164,13sleep10-21swapper/107:08:551
1273728168,9sleep30-21swapper/307:08:133
206842520,0sleep30-21swapper/310:55:203
12977993029,1cyclictest0-21swapper/307:26:423
12975992922,6cyclictest0-21swapper/212:26:332
1297799281,9cyclictest7009-21munin-node12:05:103
1297099241,2cyclictest141rcu_preempt11:34:381
1297599230,2cyclictest0-21swapper/211:00:242
12977992121,0cyclictest0-21swapper/311:31:403
12970992114,3cyclictest7129-21ssh09:30:091
1297799202,1cyclictest5767-21diskmemload09:50:293
1297799200,1cyclictest0-21swapper/310:16:213
1296799200,2cyclictest0-21swapper/011:31:410
1296799200,20cyclictest0-21swapper/009:51:180
1296799200,19cyclictest0-21swapper/010:55:140
12977991918,1cyclictest0-21swapper/311:02:323
1297799190,2cyclictest0-21swapper/311:26:073
1297799190,13cyclictest0-21swapper/311:39:033
1297599193,16cyclictest0-21swapper/207:58:532
1297599193,15cyclictest2913-21diskmemload10:05:352
12975991916,3cyclictest0-21swapper/212:16:532
12967991918,1cyclictest0-21swapper/011:10:420
1296799190,17cyclictest0-21swapper/009:30:340
1297799189,3cyclictest19545-21wc07:20:253
12977991816,1cyclictest0-21swapper/311:20:143
1297099180,1cyclictest0-21swapper/110:44:491
12967991816,1cyclictest0-21swapper/011:40:130
1297799179,2cyclictest8087-21sed09:30:163
1297799172,5cyclictest0-21swapper/312:30:003
12977991716,1cyclictest0-21swapper/310:34:343
12977991710,2cyclictest7092-21sed09:10:163
12977991710,2cyclictest30496-21grep08:50:263
12977991710,2cyclictest15332-21grep08:20:163
12975991716,1cyclictest0-21swapper/208:14:472
12975991715,2cyclictest32-21ksoftirqd/211:33:042
12975991714,3cyclictest0-21swapper/212:04:132
12975991714,3cyclictest0-21swapper/209:19:532
1297099171,1cyclictest5767-21diskmemload11:13:221
1297099170,1cyclictest0-21swapper/109:44:371
223372160,1chrt0-21swapper/212:14:162
1297799160,15cyclictest0-21swapper/309:19:123
1297599162,12cyclictest8619-21mailstats12:05:212
12975991611,5cyclictest4958-21apache211:37:522
1297099161,15cyclictest5767-21diskmemload11:16:491
12970991611,3cyclictest18571-21sort07:20:151
1297099160,1cyclictest5767-21diskmemload12:13:511
1296799164,10cyclictest0-21swapper/008:40:130
12967991616,0cyclictest0-21swapper/010:53:090
110722160,0sleep20-21swapper/211:09:372
1297799156,5cyclictest29480-21runrttasks10:40:533
12977991514,1cyclictest0-21swapper/307:15:133
1297799150,2cyclictest0-21swapper/311:15:413
12975991514,1cyclictest0-21swapper/210:54:492
12975991514,1cyclictest0-21swapper/208:50:142
1297599150,13cyclictest0-21swapper/210:37:542
1297599150,12cyclictest0-21swapper/211:55:492
12970991511,3cyclictest19385-21basename08:30:121
12970991511,3cyclictest19385-21basename08:30:121
1297099150,13cyclictest0-21swapper/112:15:011
1297099150,0cyclictest0-21swapper/110:55:581
1296799150,1cyclictest0-21swapper/010:45:140
1297799140,1cyclictest0-21swapper/310:28:463
1297599143,10cyclictest0-21swapper/211:50:592
12975991411,2cyclictest23365-21cron09:40:002
12975991410,2cyclictest13508-21tr07:10:282
1297599140,1cyclictest0-21swapper/211:29:512
1297099145,2cyclictest225-21systemd-journal07:14:561
1297099143,3cyclictest27924-21apt-get11:00:111
12970991412,1cyclictest0-21swapper/111:21:211
12970991411,2cyclictest9034-21head12:05:251
12970991411,2cyclictest32628-21cron10:05:001
12970991411,2cyclictest24893-21cut08:40:151
12970991410,2cyclictest0-21swapper/109:15:111
1297099140,2cyclictest0-21swapper/111:55:141
1297099140,1cyclictest0-21swapper/109:49:261
1297099140,14cyclictest0-21swapper/112:34:261
1297099140,11cyclictest0-21swapper/111:25:021
1296799147,4cyclictest520-21gmain10:20:140
1296799145,8cyclictest4179-21perf11:25:000
1296799142,6cyclictest29820-21munin-run07:45:000
12967991410,3cyclictest24505-21cat07:30:260
1296799140,4cyclictest0-21swapper/011:05:010
1296799140,4cyclictest0-21swapper/009:47:370
1296799140,4cyclictest0-21swapper/008:50:140
1296799140,12cyclictest22826-21cut08:35:190
1297799139,3cyclictest2471-21grep07:55:143
1297799139,2cyclictest3464-21wc09:05:013
1297799135,4cyclictest225-21systemd-journal08:05:003
1297799134,8cyclictest5767-21diskmemload12:18:473
12977991310,2cyclictest27976-21tr07:40:143
12977991310,2cyclictest18256-21wc12:30:243
1297799130,12cyclictest5579-21grep11:05:243
12975991312,1cyclictest0-21swapper/209:52:262
12975991312,1cyclictest0-21swapper/209:44:322
1297599131,1cyclictest0-21swapper/210:20:132
12975991310,2cyclictest418-21cron12:40:002
12975991310,2cyclictest32414-21tr12:20:132
12975991310,2cyclictest3069-21grep10:45:142
12975991310,2cyclictest28733-21sort07:40:182
12975991310,2cyclictest26772-21tr08:45:132
12975991310,2cyclictest20096-21grep08:30:152
12975991310,2cyclictest20096-21grep08:30:152
1297599130,13cyclictest0-21swapper/208:23:192
1297099139,3cyclictest638-21grep07:50:161
1297099139,3cyclictest11402-21sed10:30:191
1297099132,6cyclictest9876-21latency_hist10:30:001
1297099131,1cyclictest5767-21diskmemload11:54:071
12970991311,2cyclictest0-21swapper/110:22:051
12970991310,2cyclictest8978-21grep12:25:191
12970991310,2cyclictest30803-21grep07:45:161
12970991310,2cyclictest22622-21apt-config07:30:001
12970991310,2cyclictest22285-21sed11:35:161
12970991310,2cyclictest1966-21sed09:00:171
12970991310,2cyclictest14724-21seq08:20:121
1297099130,2cyclictest0-21swapper/112:20:521
1296799131,8cyclictest14158-21aten2_r7power_c11:50:130
12967991310,2cyclictest26467-21head12:35:210
1296799130,12cyclictest8849-21wc12:05:250
1297799129,2cyclictest4449-21ssh10:45:233
1297799129,2cyclictest32443-21grep12:20:133
1297799129,2cyclictest31492-21wc07:45:253
1297799129,2cyclictest24268-21cut09:20:213
1297799129,2cyclictest22981-21grep11:55:153
1297799129,2cyclictest17242-21grep08:25:153
1297799129,2cyclictest14350-21basename08:20:003
1297799129,2cyclictest1040-21wc09:00:013
1297799128,3cyclictest26768-21grep10:20:173
1297799128,3cyclictest25804-21grep09:40:263
1297799128,3cyclictest1205-21head09:45:213
1297799121,9cyclictest25671-21df_abs12:35:153
12977991212,0cyclictest0-21swapper/310:54:013
12977991211,0cyclictest0-21swapper/311:50:123
1297599129,2cyclictest5261-21grep09:05:262
1297599129,2cyclictest32585-21cut07:50:132
1297599129,2cyclictest31718-21df08:55:152
1297599129,2cyclictest30648-21head11:40:172
1297599129,2cyclictest2853-21sed09:00:262
1297599129,2cyclictest22263-21grep08:35:142
1297599129,2cyclictest17737-21cat07:20:002
1297599129,2cyclictest175782sleep210:15:122
1297599129,2cyclictest10134-21head10:10:162
1297599128,3cyclictest30350-21tr07:45:142
1297599128,3cyclictest29414-21grep11:20:142
1297599126,3cyclictest225-21systemd-journal11:50:002
12975991211,1cyclictest0-21swapper/209:30:132
1297599120,11cyclictest2530-21df10:25:162
1297099129,2cyclictest30156-21cstates11:40:151
1297099129,2cyclictest24725-21cat09:20:251
1297099129,2cyclictest10523-21grep08:10:171
1297099129,2cyclictest10123-21grep10:10:161
1297099128,3cyclictest32617-21head09:25:221
1297099121,8cyclictest8072-21kworker/1:110:35:131
12970991210,2cyclictest0-21swapper/110:48:241
1297099120,11cyclictest31396-21tr12:00:131
1297099120,11cyclictest25749-21cut12:35:161
1297099120,11cyclictest18931-21ssh10:54:551
1296799129,2cyclictest7094-21gltestperf09:10:160
1296799129,2cyclictest6654-21grep11:25:270
1296799129,2cyclictest5598-21grep08:00:170
1296799129,2cyclictest30709-21cat12:00:010
1296799129,2cyclictest23207-21ssh11:35:250
1296799129,2cyclictest20911-21cat08:30:240
1296799129,2cyclictest20911-21cat08:30:240
1296799129,2cyclictest19485-21sed10:15:250
1296799129,2cyclictest16331-21grep07:15:170
1296799129,2cyclictest16311-21sed08:20:260
1296799129,2cyclictest1114-21sed07:50:190
1296799129,2cyclictest10763-21wc10:10:210
1296799128,3cyclictest993-21grep10:05:140
1296799128,3cyclictest8557-21cut08:05:200
1296799128,3cyclictest25056-21ssh09:40:180
1296799127,4cyclictest1436-21sendmail-mta12:02:470
1296799121,7cyclictest0-21swapper/010:02:410
1296799120,7cyclictest24549-21cron12:35:000
1296799120,2cyclictest0-21swapper/010:41:320
1297799118,2cyclictest9069-21ssh10:10:033
1297799118,2cyclictest8426-21sed08:05:193
1297799118,2cyclictest695-21grep07:50:163
1297799118,2cyclictest5624-21ssh10:07:403
1297799118,2cyclictest31454-21ssh10:04:053
1297799118,2cyclictest31051-21ssh09:25:043
1297799118,2cyclictest24801-21tune2fs08:40:163
1297799118,2cyclictest23936-21cat07:30:193
1297799118,2cyclictest23499-21taskset08:35:583
1297799118,2cyclictest18657-21ssh12:11:243
1297799118,2cyclictest17529-21ssh11:13:083
1297799118,2cyclictest16833-21tr09:55:163
1297799117,2cyclictest11363-21wc08:10:263
1297799116,2cyclictest225-21systemd-journal09:10:013
1297799115,2cyclictest225-21systemd-journal11:45:003
1297799113,5cyclictest5767-21diskmemload09:35:363
1297799111,3cyclictest141rcu_preempt12:00:593
1297799110,3cyclictest0-21swapper/310:35:123
1297799110,10cyclictest32180-21sh11:40:433
1297599118,2cyclictest8783-21ssh09:10:522
1297599118,2cyclictest32053-21head09:25:152
1297599118,2cyclictest32000-21ssh09:45:032
1297599118,2cyclictest31183-21ssh10:42:132
1297599118,2cyclictest24795-21rm10:00:112
1297599118,2cyclictest23560-21grep09:20:152
1297599118,2cyclictest22035-21ssh12:33:032
1297599118,2cyclictest20971-21grep10:55:222
1297599118,2cyclictest12553-21expr11:10:162
1297599118,2cyclictest11761-21rt-features10:30:222
1297599117,3cyclictest9094-21sed08:05:252
1297599110,2cyclictest0-21swapper/207:25:132
1297599110,10cyclictest22936-21ssh11:15:522
1297599110,10cyclictest0-21swapper/208:40:172
1297099118,2cyclictest9926-21ssh11:08:431
1297099118,2cyclictest23980-21cat07:30:191
1297099118,2cyclictest23435-21ssh09:59:201
1297099118,2cyclictest23369-21sed08:35:251
1297099118,2cyclictest20469-21seq10:16:061
1297099118,2cyclictest13196-21ssh11:49:411
1297099118,2cyclictest12481-21grep08:15:161
1297099118,2cyclictest11249-21ssh09:12:491
1297099117,4cyclictest27323-21taskset07:38:581
1297099117,3cyclictest6918-21taskset08:04:311
1297099117,3cyclictest5744-21cron09:10:001
1297099117,3cyclictest3341-21ssh10:05:561
1297099111,6cyclictest225-21systemd-journal07:45:001
1297099110,10cyclictest14257-21ssh09:53:391
1296799118,2cyclictest8641-21ssh11:46:060
1296799118,2cyclictest4372-21hddtemp_smartct09:05:170
1296799118,2cyclictest31115-21cat07:45:190
1296799118,2cyclictest31006-21ssh11:01:210
1296799118,2cyclictest28144-21grep08:45:240
1296799118,2cyclictest27308-21ssh11:19:180
1296799118,2cyclictest25828-21ssh09:21:100
1296799118,2cyclictest25322-21ssh12:15:220
1296799118,2cyclictest23306-21ssh10:37:360
1296799118,2cyclictest2130-21ssh09:26:290
1296799118,2cyclictest18960-21wc07:20:180
1296799118,2cyclictest17384-21cat09:35:240
1296799118,2cyclictest15591-21ssh12:29:410
1296799117,4cyclictest4958-21apache212:11:470
1296799117,2cyclictest25566-21tr07:35:140
1296799111,6cyclictest225-21systemd-journal10:30:000
1296799110,9cyclictest0-21swapper/009:16:470
1296799110,5cyclictest0-21swapper/007:55:130
1296799110,10cyclictest23817-21chrt09:59:390
1297799108,1cyclictest28686-21cat08:50:013
1297799108,1cyclictest14959-21chrt07:10:413
1297799107,2cyclictest25545-21expr07:35:143
1297599107,2cyclictest5085-21cut08:00:152
1297599107,2cyclictest23557-21grep07:30:162
1297599107,2cyclictest18231-21wc08:25:222
1297599107,2cyclictest17978-21dpkg07:20:122
1297599100,9cyclictest25308-21apache_volume07:35:122
1297599100,9cyclictest16760-21cat09:55:162
1297599100,9cyclictest12773-21dnf08:15:172
1297099108,1cyclictest309372sleep108:54:481
1297099108,1cyclictest17388-21expr08:25:151
1297099108,1cyclictest16460-21sed09:35:161
1297099107,2cyclictest26667-21cpu08:45:131
1297099107,2cyclictest17339-21seq07:15:421
1296799108,1cyclictest31920-21ssh12:20:040
1296799108,1cyclictest12388-21wc08:15:160
1296799108,1cyclictest10825-21cat08:10:190
1296799107,2cyclictest31855-21grep08:55:170
1296799107,2cyclictest2922-21grep09:00:260
1296799107,2cyclictest18724-21chrt08:25:340
1296799107,2cyclictest10627-21cat10:30:140
129779997,1cyclictest212132sleep308:32:073
129779997,1cyclictest212132sleep308:32:063
129709997,1cyclictest2829-21expr07:55:151
129709995,3cyclictest8151-21grep08:05:171
129709991,4cyclictest312292sleep108:55:131
129679997,1cyclictest21269-21grep07:25:180
129679997,1cyclictest15252-21taskset07:14:260
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional