You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-27 - 16:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Fri Mar 27, 2026 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8468210279,6sleep20-21swapper/207:07:282
866029856,38sleep00-21swapper/007:09:540
654029169,17sleep10-21swapper/107:05:091
844428370,9sleep30-21swapper/307:07:093
135432610,0sleep20-21swapper/211:10:462
8765995516,35cyclictest0-21swapper/309:43:483
8765995516,35cyclictest0-21swapper/309:43:473
8765994119,21cyclictest0-21swapper/311:34:363
8765994119,21cyclictest0-21swapper/310:15:243
8765994119,21cyclictest0-21swapper/310:14:583
8765994119,21cyclictest0-21swapper/309:22:443
8759994110,27cyclictest0-21swapper/111:48:381
8765994019,20cyclictest0-21swapper/312:12:053
8765994019,20cyclictest0-21swapper/311:12:333
8765994019,20cyclictest0-21swapper/310:53:103
8765994019,20cyclictest0-21swapper/310:29:523
8765994019,20cyclictest0-21swapper/309:53:543
8765994018,21cyclictest0-21swapper/312:25:233
8765994018,21cyclictest0-21swapper/311:20:143
8765994018,21cyclictest0-21swapper/310:45:353
8765994018,21cyclictest0-21swapper/310:00:003
8765994016,23cyclictest0-21swapper/311:06:343
8759994012,24cyclictest0-21swapper/111:50:511
8759994012,24cyclictest0-21swapper/111:13:161
8765993918,21cyclictest0-21swapper/310:41:223
8765993918,21cyclictest0-21swapper/310:20:533
8765993918,20cyclictest0-21swapper/312:05:153
8765993918,20cyclictest0-21swapper/311:50:183
8765993918,20cyclictest0-21swapper/311:40:153
8765993918,20cyclictest0-21swapper/311:17:423
8765993918,20cyclictest0-21swapper/311:00:223
8765993918,20cyclictest0-21swapper/310:55:023
8765993918,20cyclictest0-21swapper/309:37:073
8765993917,21cyclictest0-21swapper/312:36:213
8765993917,21cyclictest0-21swapper/312:30:233
8765993917,21cyclictest0-21swapper/312:00:503
8765993917,21cyclictest0-21swapper/311:59:023
8765993917,21cyclictest0-21swapper/311:45:553
8765993917,21cyclictest0-21swapper/311:37:393
8765993917,21cyclictest0-21swapper/311:25:233
8765993917,21cyclictest0-21swapper/310:08:483
8765993917,21cyclictest0-21swapper/309:16:073
8765993917,21cyclictest0-21swapper/309:13:213
8765993915,23cyclictest0-21swapper/309:27:443
8765993817,21cyclictest0-21swapper/310:30:473
8765993817,20cyclictest0-21swapper/312:15:113
8765993817,20cyclictest0-21swapper/310:01:033
8765993816,21cyclictest0-21swapper/312:20:083
8765993816,21cyclictest0-21swapper/310:35:033
8765993816,21cyclictest0-21swapper/309:48:333
8765993816,21cyclictest0-21swapper/309:30:243
8759993810,24cyclictest0-21swapper/109:47:281
8759993719,17cyclictest0-21swapper/111:05:101
8759993717,19cyclictest0-21swapper/109:50:311
8759993710,24cyclictest0-21swapper/110:09:201
8759993619,17cyclictest0-21swapper/111:03:221
8759993619,16cyclictest0-21swapper/109:40:361
8759993619,16cyclictest0-21swapper/109:40:351
8759993619,16cyclictest0-21swapper/109:37:451
8759993618,17cyclictest0-21swapper/112:25:571
8759993618,17cyclictest0-21swapper/112:06:581
8759993618,17cyclictest0-21swapper/111:38:571
8759993617,18cyclictest0-21swapper/111:30:071
8759993617,18cyclictest0-21swapper/111:22:141
8759993617,18cyclictest0-21swapper/110:39:421
8759993617,18cyclictest0-21swapper/109:26:581
8759993616,19cyclictest0-21swapper/110:02:311
8759993615,20cyclictest0-21swapper/112:21:031
8759993615,20cyclictest0-21swapper/110:55:141
8759993614,21cyclictest0-21swapper/112:38:501
8759993614,21cyclictest0-21swapper/112:33:001
875999360,34cyclictest0-21swapper/109:17:171
8759993521,13cyclictest0-21swapper/112:10:151
8759993520,14cyclictest0-21swapper/110:40:521
8759993519,15cyclictest0-21swapper/111:40:061
8759993518,16cyclictest0-21swapper/111:17:491
8759993518,16cyclictest0-21swapper/110:50:031
8759993518,16cyclictest0-21swapper/110:22:261
8759993518,16cyclictest0-21swapper/110:11:591
8759993518,16cyclictest0-21swapper/109:23:371
8759993517,17cyclictest0-21swapper/110:25:231
8759993517,17cyclictest0-21swapper/110:17:221
8759993516,18cyclictest0-21swapper/112:00:051
8759993516,18cyclictest0-21swapper/110:45:051
8759993516,18cyclictest0-21swapper/110:30:051
8759993516,18cyclictest0-21swapper/109:55:131
8759993511,23cyclictest0-21swapper/109:10:331
875999350,34cyclictest0-21swapper/112:18:021
875999350,34cyclictest0-21swapper/111:55:081
875999350,34cyclictest0-21swapper/111:25:101
875999350,34cyclictest0-21swapper/109:30:321
8765992718,8cyclictest0-21swapper/309:10:003
876199270,23cyclictest291irq_work/211:18:352
876199250,2cyclictest0-21swapper/212:00:572
875699250,2cyclictest0-21swapper/009:12:370
876199230,4cyclictest0-21swapper/212:28:122
876199230,23cyclictest0-21swapper/212:14:212
875699230,23cyclictest0-21swapper/011:51:380
8761992220,1cyclictest32-21ksoftirqd/212:22:082
876199220,4cyclictest0-21swapper/212:16:552
876199220,22cyclictest0-21swapper/209:38:542
8759992218,3cyclictest10244-21cut08:20:121
875699220,2cyclictest0-21swapper/011:18:480
199892220,21sleep00-21swapper/011:34:460
876199210,2cyclictest0-21swapper/210:55:112
876199210,18cyclictest0-21swapper/209:40:492
876199210,18cyclictest0-21swapper/209:40:492
875999212,15cyclictest25-21ksoftirqd/107:35:271
875699210,1cyclictest0-21swapper/010:55:060
876199202,3cyclictest141rcu_preempt09:15:582
8761991911,8cyclictest32-21ksoftirqd/211:58:162
876199190,18cyclictest0-21swapper/210:41:332
876199190,18cyclictest0-21swapper/210:09:392
876199181,2cyclictest0-21swapper/212:30:562
876199180,6cyclictest0-21swapper/211:05:122
875699180,1cyclictest0-21swapper/009:35:560
8761991716,1cyclictest0-21swapper/211:48:322
8761991716,1cyclictest0-21swapper/210:26:052
8761991712,4cyclictest0-21swapper/209:00:132
876199170,2cyclictest0-21swapper/209:55:502
876199170,17cyclictest0-21swapper/210:01:342
876199170,16cyclictest0-21swapper/212:08:562
876199170,15cyclictest0-21swapper/210:54:232
875699170,13cyclictest0-21swapper/010:54:120
876199161,14cyclictest0-21swapper/209:46:362
8761991611,2cyclictest29047-21sh11:40:002
876199160,16cyclictest0-21swapper/209:28:572
876199160,11cyclictest141rcu_preempt09:50:082
875699169,5cyclictest475-21in:imuxsock09:30:000
875699163,5cyclictest0-21swapper/011:05:120
875699160,1cyclictest0-21swapper/009:40:490
875699160,1cyclictest0-21swapper/009:40:480
875699160,15cyclictest0-21swapper/010:01:340
8765991511,3cyclictest11965-21uniq07:15:153
876199154,7cyclictest225-21systemd-journal08:43:312
8756991514,1cyclictest0-21swapper/012:35:240
8756991514,1cyclictest0-21swapper/010:20:020
875699151,3cyclictest141rcu_preempt10:37:200
8756991511,3cyclictest13-21ksoftirqd/010:10:140
875699150,8cyclictest18323-21basename07:30:010
875699150,4cyclictest0-21swapper/011:40:180
875699150,4cyclictest0-21swapper/010:30:190
875699150,11cyclictest0-21swapper/010:15:090
875699150,10cyclictest0-21swapper/010:45:400
8765991411,2cyclictest9586-21uniq07:10:153
8765991410,3cyclictest9725-21cat08:20:003
876199146,1cyclictest141rcu_preempt09:22:292
876199140,4cyclictest0-21swapper/210:48:352
876199140,3cyclictest0-21swapper/210:20:022
875999140,2cyclictest0-21swapper/108:38:331
8756991413,1cyclictest0-21swapper/012:03:370
8756991410,1cyclictest13-21ksoftirqd/009:16:440
875699140,6cyclictest0-21swapper/011:12:100
875699140,2cyclictest0-21swapper/010:27:210
875699140,1cyclictest0-21swapper/012:08:560
8765991310,2cyclictest24928-21uniq08:50:163
8765991310,2cyclictest22167-21head07:35:233
8765991310,2cyclictest13087-21grep08:25:153
876199136,6cyclictest141rcu_preempt09:34:162
876199134,7cyclictest0-21swapper/208:25:012
8761991310,2cyclictest23224-21wc08:45:202
8761991310,2cyclictest18136-21wc10:35:212
876199130,2cyclictest0-21swapper/210:10:542
876199130,1cyclictest0-21swapper/211:26:512
875999139,3cyclictest13512-21basename07:19:591
8759991310,2cyclictest27683-21wc08:55:191
8756991312,1cyclictest17794-21apache210:05:110
8756991310,2cyclictest3549-21ssh11:25:000
875699130,1cyclictest0-21swapper/012:25:130
875699130,12cyclictest0-21swapper/009:53:240
875699130,10cyclictest0-21swapper/010:40:170
876599129,2cyclictest3301-21grep08:05:163
876599129,2cyclictest19654-21grep07:30:203
876599129,2cyclictest15872-21sh07:25:003
876199129,2cyclictest24954-21sort08:50:162
876199129,2cyclictest21643-21grep07:35:162
876199129,2cyclictest19301-21expr07:30:162
876199129,2cyclictest14947-21tr08:30:142
8761991211,1cyclictest0-21swapper/211:32:082
876199120,8cyclictest0-21swapper/211:01:252
876199120,6cyclictest30601-21irqstats11:40:182
876199120,6cyclictest0-21swapper/210:18:312
876199120,2cyclictest0-21swapper/211:20:192
876199120,11cyclictest125902sleep210:32:192
875999129,2cyclictest23616-21grep08:45:241
875999129,2cyclictest14018-21tr07:20:141
875999128,2cyclictest9020-21cut07:10:141
875999127,4cyclictest9557-21chrt08:18:381
875699129,2cyclictest442-21ssh12:20:110
875699129,2cyclictest30130-21ssh11:01:330
875699129,2cyclictest27271-21ssh11:38:400
875699129,2cyclictest22673-21tail08:45:150
875699129,2cyclictest21706-21cat08:45:000
875699129,2cyclictest2084-21ssh09:47:490
875699129,2cyclictest16040-21date07:25:000
875699126,6cyclictest0-21swapper/012:16:190
8756991210,1cyclictest15899-21ls08:30:210
875699120,8cyclictest0-21swapper/009:21:230
875699120,3cyclictest0-21swapper/009:35:000
876599119,1cyclictest26463-21cut07:45:163
876599118,2cyclictest30778-21expr07:55:153
876599118,2cyclictest29937-21sort09:00:173
876599118,2cyclictest28303-21sort07:50:153
876599118,2cyclictest28303-21sort07:50:153
876599112,2cyclictest0-21swapper/308:00:113
876599110,10cyclictest23255-21chrt08:45:223
876199119,2cyclictest12495-21awk07:15:192
876199119,1cyclictest5933-21cut08:10:182
876199119,1cyclictest28368-21grep08:55:262
876199119,1cyclictest1795-21sed09:10:012
876199119,1cyclictest17105-21cat07:25:192
876199118,2cyclictest29674-21cat07:50:272
876199118,2cyclictest29674-21cat07:50:262
876199118,2cyclictest26043-21sh12:35:102
876199118,2cyclictest12591-21tr08:25:142
876199117,1cyclictest141rcu_preempt08:38:332
876199115,1cyclictest32-21ksoftirqd/209:10:222
876199112,7cyclictest14212-21sh11:50:122
875999118,2cyclictest4037-21tr08:05:211
875999118,2cyclictest31875-21grep07:55:251
875999118,2cyclictest16729-21taskset08:34:051
875999118,2cyclictest1094-21tail09:05:261
875999110,1cyclictest0-21swapper/108:50:151
875699119,2cyclictest9461-21expr07:10:150
875699119,2cyclictest18983-21cat07:30:140
875699119,1cyclictest661-21tr08:00:130
875699119,1cyclictest31238-21tail07:55:170
875699119,1cyclictest27802sleep008:05:110
875699119,1cyclictest11426-21hostname07:15:130
875699118,2cyclictest7282-21ssh11:45:250
875699118,2cyclictest5634-21expr08:10:160
875699118,2cyclictest30748-21http12:00:010
875699118,2cyclictest27011-21expr08:55:160
875699118,2cyclictest13860-21ssh09:55:120
875699116,1cyclictest13-21ksoftirqd/007:40:010
875699115,1cyclictest141rcu_preempt12:33:060
875699113,3cyclictest13-21ksoftirqd/011:26:330
875699112,7cyclictest2218-21kworker/0:108:50:130
875699110,11cyclictest18935-21taskset08:36:100
875699110,10cyclictest28173-21aten2_r7power_p07:50:150
875699110,10cyclictest28173-21aten2_r7power_p07:50:140
875699110,10cyclictest17083-21ssh12:10:190
876599108,1cyclictest15681-21sed08:30:193
876599107,2cyclictest26795-21grep08:55:143
876599107,2cyclictest23639-21cat07:40:143
876599107,2cyclictest19914-21expr08:40:143
876599107,2cyclictest18354-21grep08:35:213
876599100,9cyclictest11966-21taskset08:23:513
876199109,1cyclictest31435-21sed07:55:192
876199109,1cyclictest10847-21chrt07:12:012
876199108,1cyclictest2918-21grep08:05:132
876199107,2cyclictest8336-21expr08:15:162
876199107,2cyclictest26628-21uname07:45:182
876199104,5cyclictest32-21ksoftirqd/207:25:012
876199102,5cyclictest0-21swapper/207:40:132
875999108,1cyclictest766-21expr08:00:131
875999108,1cyclictest0-21swapper/107:45:121
875999107,2cyclictest30889-21sed09:00:251
875999107,2cyclictest19289-21expr07:30:161
875999104,4cyclictest5152-21/usr/sbin/munin08:10:201
875999100,9cyclictest29635-21sed07:50:271
875999100,9cyclictest29635-21sed07:50:261
875699109,1cyclictest31848-21apache_accesses09:05:120
875699109,1cyclictest23876-21df_abs07:40:160
875699109,1cyclictest12641-21cstates08:25:140
875699108,1cyclictest30970-21grep09:00:260
875699108,1cyclictest25798-21aten2_r7power_p07:45:120
875699108,1cyclictest10302-21cat08:20:120
875699103,2cyclictest0-21swapper/008:15:130
87659997,1cyclictest16427-21cut07:25:143
87659991,1cyclictest0-21swapper/308:10:143
87619998,1cyclictest864-21cat08:00:122
87599997,1cyclictest16769-21cut07:25:171
87599996,2cyclictest20854-21wc08:40:221
87599996,2cyclictest12586-21tr08:25:141
87599991,4cyclictest225-21systemd-journal07:45:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional