You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-07 - 05:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Mon Sep 07, 2026 00:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1010329369,8sleep00-21swapper/019:06:210
1017229171,16sleep20-21swapper/219:07:152
1012428164,13sleep10-21swapper/119:06:381
1005228168,9sleep30-21swapper/319:05:453
323252590,0sleep30-21swapper/321:26:243
239372300,0sleep30-21swapper/321:21:233
1046699270,26cyclictest0-21swapper/222:01:182
1046699242,12cyclictest0-21swapper/220:15:132
1046699230,23cyclictest0-21swapper/221:39:262
1046699230,22cyclictest0-21swapper/221:56:362
1045799230,23cyclictest0-21swapper/023:54:540
1045799222,2cyclictest0-21swapper/022:45:130
319312210,2sleep131276-21kworker/1:122:43:441
1046299210,2cyclictest0-21swapper/121:36:471
1046299210,0cyclictest0-21swapper/122:21:001
1046699200,20cyclictest0-21swapper/223:29:102
1046699200,1cyclictest0-21swapper/222:05:132
10462992018,2cyclictest25-21ksoftirqd/122:04:011
304632190,0chrt0-21swapper/023:01:420
1046699190,2cyclictest0-21swapper/223:54:082
10457991911,7cyclictest29960-21apache222:05:060
1046299184,7cyclictest11665-21/usr/sbin/munin22:50:271
1046299180,2cyclictest0-21swapper/122:38:031
1046999176,2cyclictest13300-21taskset22:51:273
1046999170,17cyclictest0-21swapper/321:37:213
1045799172,4cyclictest225-21systemd-journal22:15:000
10457991710,6cyclictest1109-21perf21:04:590
1045799170,0cyclictest0-21swapper/022:35:260
160812160,0sleep00-21swapper/021:16:480
1046699160,15cyclictest0-21swapper/222:30:072
1046699160,14cyclictest0-21swapper/222:58:332
10462991614,1cyclictest0-21swapper/123:35:131
1046299160,3cyclictest0-21swapper/123:30:201
1046299160,10cyclictest0-21swapper/121:55:111
1045799164,6cyclictest6998-21awk00:24:590
1045799160,15cyclictest0-21swapper/022:01:090
1046999154,2cyclictest30115-21apache_processe00:00:143
10469991513,2cyclictest39-21ksoftirqd/300:26:353
1046999150,15cyclictest0-21swapper/323:39:353
1046999150,14cyclictest0-21swapper/322:38:573
10462991510,2cyclictest0-21swapper/123:52:351
1046299150,2cyclictest0-21swapper/120:00:161
1045799150,14cyclictest0-21swapper/022:30:380
10469991410,3cyclictest17353-21tr20:30:153
10466991414,0cyclictest0-21swapper/223:10:562
10466991410,3cyclictest23759-21turbostat00:34:592
10466991410,3cyclictest17827-21cut20:30:172
1046699140,1cyclictest0-21swapper/223:59:222
1046699140,13cyclictest0-21swapper/222:40:552
1046299147,6cyclictest17585-21sed22:15:211
1046299146,5cyclictest38150irq/131-snd_hda00:03:201
1046299141,12cyclictest0-21swapper/120:15:131
10462991411,1cyclictest55550irq/126-enp2s0-23:25:061
1045799147,6cyclictest29960-21apache221:38:290
10457991411,2cyclictest29688-21grep23:40:170
1045799140,1cyclictest0-21swapper/022:43:040
1045799140,14cyclictest0-21swapper/000:37:050
256792130,0chrt0-21swapper/000:15:480
1046999139,3cyclictest2730-21cat22:45:183
1046999139,3cyclictest14770-21apache_accesses21:55:123
10469991310,2cyclictest31109-21cron20:59:593
10469991310,2cyclictest28765-21ssh22:03:133
10469991310,2cyclictest26216-21cron22:59:593
10469991310,2cyclictest23048-21ls20:40:213
10469991310,2cyclictest17180-21sed19:20:253
10469991310,2cyclictest14158-21cron00:10:003
1046999130,13cyclictest0-21swapper/321:52:053
1046699139,3cyclictest704-21wc19:55:202
1046699139,3cyclictest704-21wc19:55:192
1046699139,3cyclictest21139-21grep20:35:272
1046699134,5cyclictest0-21swapper/219:30:142
1046699131,6cyclictest0-21swapper/200:06:562
10466991310,2cyclictest8983-21wc22:10:222
10466991310,2cyclictest31426-21ntpq00:00:222
10466991310,2cyclictest26237-21tr19:40:232
1046699130,12cyclictest0-21swapper/200:25:322
1046699130,12cyclictest0-21swapper/200:10:112
1046299139,2cyclictest17086-21sed19:20:241
1046299138,4cyclictest29405-21chrt19:49:131
1046299138,3cyclictest1142-21runrttasks21:53:221
1046299137,1cyclictest684-21needreboot22:05:201
10462991311,1cyclictest3472-21sh21:10:001
1046299131,0cyclictest141rcu_preempt00:21:171
10462991310,2cyclictest30663-21grep20:55:261
10462991310,2cyclictest24244-21grep20:45:141
1046299130,1cyclictest0-21swapper/123:16:251
1046299130,1cyclictest0-21swapper/100:28:501
1046299130,0cyclictest0-21swapper/122:27:261
1045799139,3cyclictest20715-21ssh23:35:130
1045799139,2cyclictest29468-21cron19:50:000
10457991312,1cyclictest0-21swapper/021:14:560
10457991310,2cyclictest31516-21grep19:50:260
10457991310,2cyclictest21093-21grep20:35:260
10457991310,2cyclictest20337-21sed23:15:180
1046999129,2cyclictest8265-21tail20:10:183
1046999129,2cyclictest6928-21sed23:45:253
1046999129,2cyclictest28712-21grep23:20:173
1046999129,2cyclictest27860-21chrt20:50:233
1046999129,2cyclictest24844-21rm00:15:243
1046999129,2cyclictest24081-21cat19:35:263
1046999129,2cyclictest16946-21wc20:30:003
1046999129,2cyclictest14760-21head23:50:223
1046999129,2cyclictest1415-21cut22:25:163
1046999129,2cyclictest13918-21cat23:30:263
1046999128,3cyclictest1632-21cut21:05:143
1046999128,3cyclictest13049-21grep20:20:193
1046999123,6cyclictest3480-21diskmemload22:10:193
1046999123,5cyclictest6986-21munin-plugin-st00:24:593
1046999123,4cyclictest0-21swapper/321:40:163
10469991210,1cyclictest28100-21grep19:45:173
1046999120,11cyclictest19308-21kworker/u8:222:41:263
1046999120,11cyclictest0-21swapper/300:13:123
1046699129,2cyclictest7156-21grep21:30:292
1046699129,2cyclictest32669-21wc21:00:232
1046699129,2cyclictest31033-21cron00:19:592
1046699129,2cyclictest27491-21grep23:00:192
1046699129,2cyclictest2654-21tr20:00:162
1046699129,2cyclictest2536-21sed22:25:262
1046699129,2cyclictest23530-21head20:40:262
1046699129,2cyclictest22994-21tail21:40:182
1046699129,2cyclictest21488-21head21:20:162
1046699129,2cyclictest18943-21cut19:25:192
1046699129,2cyclictest18138-21grep22:15:252
1046699129,2cyclictest17174-21head19:20:252
1046699128,3cyclictest7821-21tr20:10:162
1046699128,3cyclictest20552-21ssh23:35:102
1046699128,2cyclictest2320-21cut22:45:162
10466991211,1cyclictest55650irq/127-enp2s0-22:53:082
10466991210,1cyclictest32428-21head00:20:172
1046699120,11cyclictest0-21swapper/221:25:492
1046299129,2cyclictest8683-21grep22:10:201
1046299129,2cyclictest28253-21tail20:50:271
1046299129,2cyclictest16158-21strings00:10:221
1046299129,1cyclictest25-21ksoftirqd/121:40:371
1046299129,1cyclictest25-21ksoftirqd/100:05:321
1046299128,3cyclictest29764-21ls23:20:271
1046299128,3cyclictest22590-21tail21:20:261
1046299127,2cyclictest25-21ksoftirqd/100:00:001
10462991210,1cyclictest4826-21head21:10:181
10462991210,1cyclictest18604-21tr22:55:131
1046299120,11cyclictest0-21swapper/123:45:151
1046299120,11cyclictest0-21swapper/121:30:251
1045799129,2cyclictest6737-21sort00:05:180
1045799129,2cyclictest6249-21tr23:45:200
1045799129,2cyclictest5982-21sort21:30:200
1045799129,2cyclictest3651-21tail23:05:190
1045799129,2cyclictest32410-21tr19:55:150
1045799129,2cyclictest32410-21tr19:55:150
1045799129,2cyclictest25784-21grep19:40:190
1045799129,2cyclictest22432-21grep20:40:170
1045799129,2cyclictest1826-21cat22:25:200
1045799129,2cyclictest16981-21ssh00:30:190
1045799129,2cyclictest15181-21cron19:19:590
1045799129,2cyclictest12315-21wget23:30:150
1045799129,2cyclictest11407-21grep19:10:180
1045799129,2cyclictest11006-21grep20:15:200
1045799120,1cyclictest0-21swapper/023:10:430
1046999119,1cyclictest846-21head22:05:223
1046999119,1cyclictest25653-21cut00:35:233
1046999119,1cyclictest10884-21grep22:30:273
1046999118,2cyclictest8707-21ssh23:08:313
1046999118,2cyclictest4927-21cat21:30:013
1046999118,2cyclictest3788-21ssh23:25:093
1046999118,2cyclictest32457-21grep19:55:163
1046999118,2cyclictest32457-21grep19:55:153
1046999118,2cyclictest30786-21cat19:50:183
1046999118,2cyclictest29348-21ssh22:22:353
1046999118,2cyclictest28062-21strings23:00:223
1046999118,2cyclictest24161-21chrt20:45:013
1046999118,2cyclictest17877-21ssh23:14:093
1046999118,2cyclictest11455-21grep19:10:173
1046999115,5cyclictest3480-21diskmemload23:55:033
1046999113,5cyclictest4461-21cron20:05:003
1046999110,10cyclictest4722-21tune2fs21:10:173
1046699119,1cyclictest130292sleep221:15:182
1046699118,2cyclictest7390-21sed21:50:202
1046699118,2cyclictest5946-21sh21:50:012
1046699118,2cyclictest3210-21chrt21:06:482
1046699118,2cyclictest29738-21cut23:40:172
1046699118,2cyclictest28910-21chrt20:55:012
1046699118,2cyclictest27742-21ssh23:20:072
1046699118,2cyclictest27358-21grep20:50:182
1046699118,2cyclictest26569-21grep22:20:262
1046699118,2cyclictest25704-21ssh23:18:442
1046699118,2cyclictest23240-21cut19:35:172
1046699118,2cyclictest22100-21sh22:37:402
1046699118,2cyclictest16637-21chrt20:28:062
1046699118,2cyclictest14179-21sed23:30:272
1046699117,4cyclictest5215-21sed21:10:202
1046699110,11cyclictest0-21swapper/200:36:422
1046699110,10cyclictest7373-21ssh23:45:452
1046699110,10cyclictest10211-21ssh23:09:412
1046299119,1cyclictest1738-21cut22:45:131
1046299118,2cyclictest7794-21sh23:07:491
1046299118,2cyclictest7736-21tr20:10:161
1046299118,2cyclictest32422-21tr19:55:151
1046299118,2cyclictest32422-21tr19:55:151
1046299118,2cyclictest31257-21sh21:25:331
1046299118,2cyclictest29291-21sh00:37:491
1046299118,2cyclictest28392-21rm23:00:251
1046299118,2cyclictest23660-21taskset20:41:311
1046299118,2cyclictest23002-21ssh00:15:021
1046299118,2cyclictest2134-21ssh23:43:101
1046299118,2cyclictest17474-21expr20:30:151
1046299118,2cyclictest15989-21wget00:30:131
1046299118,2cyclictest15579-21cat20:25:181
1046299118,2cyclictest15031-21taskset19:18:051
1046299118,2cyclictest12661-21cat20:20:161
1046299118,2cyclictest12290-21ssh22:31:291
1046299117,3cyclictest19473-21grep20:35:141
1046299110,10cyclictest12730-21expr21:15:161
1045799118,2cyclictest9184-21sed20:10:270
1045799118,2cyclictest7061-21apt-get20:10:000
1045799118,2cyclictest7050-21ssh23:26:210
1045799118,2cyclictest4461-21cron20:05:010
1045799118,2cyclictest3844-21ssh21:29:290
1045799118,2cyclictest3198-21chrt21:06:380
1045799118,2cyclictest30348-21ssh21:45:060
1045799118,2cyclictest29074-21ssh21:44:190
1045799118,2cyclictest27916-21ssh23:20:140
1045799118,2cyclictest24770-21tr22:20:140
1045799118,2cyclictest23064-21ssh22:57:320
1045799118,2cyclictest22019-21ssh23:55:160
1045799118,2cyclictest20936-21grep19:30:180
1045799118,2cyclictest20724-21basename21:20:130
1045799118,2cyclictest17074-21ssh00:10:350
1045799118,2cyclictest14722-21sh21:55:100
1045799118,2cyclictest13092-21ssh22:12:590
1045799118,2cyclictest10078-21ssh00:25:540
1045799117,3cyclictest10327-21tr22:50:180
1045799110,1cyclictest0-21swapper/021:50:200
1045799110,10cyclictest30512-21expr00:00:170
1046999108,1cyclictest5678-21tail20:05:173
1046999108,1cyclictest30438-21sh21:45:093
1046999108,1cyclictest20204-21cat19:30:123
1046999108,1cyclictest19239-21ssh23:15:023
1046999108,1cyclictest12371-21apt21:15:143
1046999107,2cyclictest32638-21grep21:00:233
1046999107,2cyclictest26395-21python19:40:273
1046999107,2cyclictest20658-21sed20:35:223
1046999107,2cyclictest20028-21cat19:30:003
1046999107,2cyclictest17286-21ls00:30:213
1046999107,2cyclictest14192-21uname19:15:203
1046999101,4cyclictest16072-21sshd22:15:073
1046999100,9cyclictest29798-21ssh23:40:193
1046699108,1cyclictest28406-21sed19:45:182
1046699108,1cyclictest26259-21chrt20:49:242
1046699108,1cyclictest14322-21taskset20:23:502
1046699107,2cyclictest6878-21chrt20:07:492
1046699107,2cyclictest30518-21grep19:50:172
1046699107,2cyclictest13730-21cut19:15:162
1046699107,2cyclictest10937-21tr19:10:162
1046299108,1cyclictest4840-21dpkg20:05:011
1046299108,1cyclictest31425-21grep19:50:261
1046299108,1cyclictest20564-21ls19:30:151
1046299108,1cyclictest19682-21chrt19:26:331
1046299108,1cyclictest11653-21grep19:10:201
1046299108,1cyclictest11508-21expr23:10:161
1046299107,2cyclictest31668-21tr21:00:151
1046299107,2cyclictest30558-21ssh21:45:151
1046299107,2cyclictest25267-21tr19:40:161
1046299107,2cyclictest23126-21sed19:35:161
1045799108,1cyclictest29573-21grep20:55:160
1045799108,1cyclictest27005-21expr20:50:160
1045799108,1cyclictest241682sleep020:45:060
1045799108,1cyclictest189992sleep020:32:560
1045799108,1cyclictest17815-21sshd19:25:080
1045799108,1cyclictest16459-21chrt20:25:470
1045799108,1cyclictest16170-21tune2fs19:20:140
1045799108,1cyclictest14221-21taskset20:22:310
104699998,1cyclictest10355-21expr20:15:153
104579997,1cyclictest23058-21cat19:35:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional