You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-03 - 22:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sat Jan 03, 2026 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3207028976,9sleep20-21swapper/207:07:062
3226128673,9sleep00-21swapper/007:09:340
3220428564,6sleep10-21swapper/107:08:501
3112628167,9sleep30-21swapper/307:05:233
3238899222,18cyclictest14442-21ntpq11:00:221
3238399214,9cyclictest0-21swapper/010:45:120
32393991814,3cyclictest26662-21apt-get08:05:133
3238899180,17cyclictest0-21swapper/107:35:221
3238399180,2cyclictest0-21swapper/011:50:140
3239399175,6cyclictest25251-21apt-get09:10:173
3239399158,6cyclictest9432-21cron09:45:003
32390991510,3cyclictest10934-21cut10:55:152
32388991512,2cyclictest21619-21sh07:55:001
32383991514,1cyclictest0-21swapper/011:55:120
32383991510,3cyclictest31441-21grep08:15:140
3238399150,10cyclictest0-21swapper/009:20:130
3239399140,4cyclictest0-21swapper/307:35:213
3239099149,3cyclictest2151-21cut08:20:182
3239099147,3cyclictest225-21systemd-journal09:00:012
3239099146,7cyclictest10663-21basename08:40:002
3239099143,9cyclictest0-21swapper/210:20:172
32390991411,2cyclictest28615-21cut11:30:182
32390991410,3cyclictest3924-21wc09:30:272
32390991410,3cyclictest17293-21grep08:50:382
32390991410,3cyclictest15382-21cron08:50:012
32390991410,3cyclictest10580-21cut07:30:172
32390991410,2cyclictest8026-21dump-pmu-power10:50:012
32388991411,2cyclictest11067-21tr07:30:201
3238899140,7cyclictest0-21swapper/111:25:211
3238899140,13cyclictest0-21swapper/110:30:121
3238899140,13cyclictest0-21swapper/108:45:151
3238399143,5cyclictest1142-21runrttasks07:55:140
3238399141,6cyclictest15444-21awk11:05:000
32383991411,2cyclictest1884-21df08:20:150
32383991410,3cyclictest11870-21cat12:05:000
3239399139,3cyclictest4691-21cron11:50:003
3239399131,7cyclictest19530-21apt-get12:20:123
32393991310,2cyclictest5486-21grep09:35:163
32393991310,2cyclictest2304-21sh07:15:003
3239099139,3cyclictest19192-21sh12:20:002
3239099136,6cyclictest697-21head09:25:172
32390991310,2cyclictest6570-21grep11:50:262
32390991310,2cyclictest32333-21cron11:39:592
32390991310,2cyclictest20547-21wc10:05:232
32390991310,2cyclictest15438-21head07:40:172
3238899131,3cyclictest0-21swapper/112:05:131
32388991310,2cyclictest27255-21grep08:05:151
32388991310,2cyclictest23334-21tail10:10:261
32388991310,2cyclictest19203-21perf07:50:001
3238399136,6cyclictest12808-21apt-get11:00:010
3238399132,8cyclictest9457-21apt-get07:30:000
3238399131,6cyclictest14427-21munin-run07:40:000
32383991312,0cyclictest0-21swapper/009:55:150
32383991310,2cyclictest28310-21grep08:05:260
32383991310,2cyclictest16889-21tr10:00:160
3239399129,2cyclictest9935-21tr09:45:143
3239399129,2cyclictest9330-21basename08:35:173
3239399129,2cyclictest8022-21idleruntime-cro10:50:013
3239399129,2cyclictest5092-21fgrep08:25:233
3239399129,2cyclictest4526-21wc10:40:183
3239399129,2cyclictest3453-21taskset08:24:303
3239399129,2cyclictest30704-21sed08:10:273
3239399129,2cyclictest29006-21sensors09:15:233
3239399129,2cyclictest28179-21taskset10:20:283
3239399129,2cyclictest26313-21sed11:25:193
3239399129,2cyclictest22432-21chrt09:03:033
3239399129,2cyclictest21960-21cut12:25:153
3239399129,2cyclictest21960-21cut12:25:143
3239399129,2cyclictest19786-21tail11:10:243
3239399129,2cyclictest14544-21cut12:10:153
3239399129,2cyclictest13821-21head12:05:273
3239399129,2cyclictest11444-21cut10:55:173
3239399129,2cyclictest10290-21sed12:00:173
3239399120,11cyclictest1752-21chrt09:25:513
3239099129,2cyclictest9464-21turbostat.cron07:29:592
3239099129,2cyclictest9036-21grep11:55:252
3239099129,2cyclictest6927-21head08:30:162
3239099129,2cyclictest31205-21ls09:20:212
3239099129,2cyclictest27434-21cut08:05:162
3239099129,2cyclictest25497-21grep09:10:182
3239099129,2cyclictest25029-21grep08:00:172
3239099129,2cyclictest24345-21cut12:30:142
3239099129,2cyclictest2380-21cat07:15:002
3239099129,2cyclictest20273-21sed12:20:152
3239099129,2cyclictest18783-21head11:10:152
3239099129,2cyclictest13800-21df11:00:162
3239099129,2cyclictest13753-21sed09:50:282
3239099128,3cyclictest4216-21sort10:40:172
3239099127,2cyclictest10270-21chrt10:53:332
3239099122,3cyclictest225-21systemd-journal08:15:222
3239099120,10cyclictest0-21swapper/207:45:252
3238899129,2cyclictest4452-21cron09:35:001
3238899129,2cyclictest3509-21uname07:15:201
3238899129,2cyclictest25128-21cat10:15:181
3238899129,2cyclictest22995-21cut07:55:211
3238899129,2cyclictest20797-21cat10:05:261
3238899129,2cyclictest19206-21ntpq11:10:191
3238899129,2cyclictest18286-21grep12:15:241
3238899129,2cyclictest10233-21head08:35:251
3238899128,3cyclictest22089-21grep12:25:151
3238899128,3cyclictest22089-21grep12:25:151
32388991211,1cyclictest32374-21cyclictest08:30:161
32388991211,1cyclictest25-21ksoftirqd/107:25:161
3238899120,1cyclictest0-21swapper/108:13:111
3238899120,11cyclictest6046-21cut09:35:211
3238399129,2cyclictest9963-21sed10:50:280
3238399129,2cyclictest880-21sort11:40:160
3238399129,2cyclictest7069-21systemctl09:40:000
3238399129,2cyclictest3333-21sed07:15:180
3238399129,2cyclictest32617-21grep09:25:140
3238399129,2cyclictest29523-21df08:10:160
3238399129,2cyclictest29207-21grep11:30:240
3238399129,2cyclictest26856-21grep09:10:300
3238399129,2cyclictest24533-21tr08:00:150
3238399129,2cyclictest20023-21grep10:05:170
3238399129,2cyclictest19120-21grep08:55:200
3238399129,2cyclictest16236-21grep11:05:170
3238399129,2cyclictest14891-21grep08:45:250
3238399128,2cyclictest24413-21cut12:30:130
3239399119,2cyclictest32241-21chrt11:38:523
3239399119,1cyclictest9927-21grep10:50:283
3239399119,1cyclictest92692sleep307:27:473
3239399119,1cyclictest7357-21sed08:30:203
3239399119,1cyclictest7081-21ls11:54:593
3239399119,1cyclictest32653-21cut11:40:143
3239399119,1cyclictest32433-21cut08:15:193
3239399119,1cyclictest29092-21tr10:25:173
3239399119,1cyclictest23691-21chrt07:56:163
3239399119,1cyclictest21672-21cut10:10:143
3239399119,1cyclictest19567-21expr10:05:163
3239399119,1cyclictest178732sleep312:15:183
3239399119,1cyclictest16278-21uniq08:50:293
3239399119,1cyclictest12289-21cut08:40:233
3239399119,1cyclictest12049-21seq09:50:123
3239399118,2cyclictest2971-21cat07:15:163
3239399118,2cyclictest24701-21chrt09:06:393
3239399117,3cyclictest2575-21tr09:30:153
3239399116,1cyclictest7339-21apt-get11:55:123
3239399115,3cyclictest26656-21apt-get12:35:153
3239399114,2cyclictest20499-21apt-get11:15:123
3239399112,2cyclictest15418-21cron11:05:013
3239399110,10cyclictest32426-21awk10:30:233
3239099119,1cyclictest30271-21grep08:10:232
3239099119,1cyclictest0-21swapper/211:15:122
3239099118,2cyclictest6656-21grep09:35:272
3239099118,2cyclictest4877-21uname08:25:212
3239099118,2cyclictest3889-21gzip07:15:242
3239099118,2cyclictest28037-21expr09:15:152
3239099118,2cyclictest27080-21taskset11:26:192
3239099118,2cyclictest24804-21head10:15:152
3239099118,2cyclictest23828-21taskset12:27:502
3239099118,2cyclictest23828-21taskset12:27:492
3239099118,2cyclictest21130-21sed07:50:252
3239099118,2cyclictest17380-21grep11:05:282
3239099118,2cyclictest16146-21grep12:10:272
3239099118,2cyclictest15679-21tr09:55:232
3239099118,2cyclictest12887-21head12:05:172
3239099117,3cyclictest3271-21sleep10:40:012
3239099117,3cyclictest1440-21grep11:40:222
3239099110,10cyclictest4882-21wc11:50:002
3238899119,2cyclictest31994-21seq09:23:311
3238899119,1cyclictest8769-21df10:50:171
3238899119,1cyclictest6322-21expr10:45:141
3238899119,1cyclictest4925-21taskset11:50:051
3238899119,1cyclictest28954-21grep10:25:161
3238899119,1cyclictest26694-21cut10:20:151
3238899119,1cyclictest25892sleep111:45:121
3238899119,1cyclictest24466-21cpu08:00:151
3238899119,1cyclictest2116-21head08:20:171
3238899119,1cyclictest20834-21cat09:00:141
3238899119,1cyclictest18498-21grep10:00:281
3238899119,1cyclictest18291-21grep08:55:151
3238899119,1cyclictest16143-21sed09:55:281
3238899119,1cyclictest15294-21forks07:40:161
3238899119,1cyclictest12844-21taskset08:42:551
3238899118,2cyclictest547-21cat08:15:261
3238899118,2cyclictest3777-21taskset08:25:141
3238899118,2cyclictest31890-21sed11:35:271
3238899118,2cyclictest27908-21cat09:15:141
3238899118,2cyclictest27709-21cat11:30:001
3238899118,2cyclictest22199-21sed11:15:251
3238899118,2cyclictest1838-21grep11:40:261
3238899118,2cyclictest17354-21sed11:05:281
3238899118,2cyclictest12104-21cat10:55:271
3238899110,10cyclictest4295-21cut10:40:171
3238899110,10cyclictest26626-21basename12:35:131
3238399119,1cyclictest13788-21grep09:50:290
3238399118,2cyclictest7554-21tr09:40:120
3238399118,2cyclictest5027-21grep08:25:230
3238399118,2cyclictest32754-21grep07:10:150
3238399118,2cyclictest3213-21grep11:45:170
3238399118,2cyclictest30950-21grep11:35:180
3238399118,2cyclictest29534-21grep10:25:190
3238399118,2cyclictest28829-21ls09:15:210
3238399118,2cyclictest27118-21grep10:20:160
3238399118,2cyclictest26917-21sed11:25:250
3238399118,2cyclictest2611-21tr09:30:150
3238399118,2cyclictest23941-21taskset12:29:180
3238399118,2cyclictest23941-21taskset12:29:180
3238399118,2cyclictest23583-21taskset10:12:450
3238399118,2cyclictest17495-21chrt08:51:380
3238399118,2cyclictest13050-21cut12:05:190
3238399118,2cyclictest10492-21head09:45:170
3238399117,3cyclictest28852-21wc12:40:010
3239399109,1cyclictest17863-21wc11:10:003
3239399109,1cyclictest150662sleep308:45:563
3239399108,2cyclictest2531-21taskset10:35:213
3239399108,2cyclictest18299-21tr07:45:213
3239399108,1cyclictest5266-21cut07:20:153
3239399108,1cyclictest30242-21tr09:20:143
3239399108,1cyclictest25412-21grep08:00:213
3239399108,1cyclictest24298-21tr10:15:123
3239399108,1cyclictest198512sleep308:55:373
3239399108,1cyclictest10699-21cut07:30:173
3239399107,2cyclictest29190-21awk11:30:243
3239399107,2cyclictest25929-21sed12:30:263
3239399107,2cyclictest18728-21taskset10:02:093
3239399104,3cyclictest19517-21apt-get07:50:133
3239399102,2cyclictest225-21systemd-journal11:22:223
3239099108,1cyclictest32374-21cyclictest07:20:142
3239099108,1cyclictest28921-21aten2_r7power_v10:25:162
3239099108,1cyclictest22912-21cut11:20:132
3239099108,1cyclictest22097-21grep07:55:162
3239099108,1cyclictest21591-21cut09:00:192
3239099108,1cyclictest13363-21sed07:35:282
3239099108,1cyclictest12266-21sendmail_mailtr08:40:232
3239099108,1cyclictest10518-21wc09:45:172
3239099108,1cyclictest10049-21ls12:00:162
3239099107,2cyclictest8994-21grep09:40:252
3239099107,2cyclictest27396-21grep12:35:182
3239099107,2cyclictest17944-21tr10:00:222
3239099100,6cyclictest225-21systemd-journal10:10:412
3238899109,1cyclictest8416-21cat11:55:181
3238899109,1cyclictest5279-21cat07:20:151
3238899109,1cyclictest463-21cat07:10:151
3238899109,1cyclictest19939-21expr12:20:121
3238899109,1cyclictest17083-21swap08:50:371
3238899109,1cyclictest10834-21ntpq09:45:201
3238899108,2cyclictest24317-21taskset12:30:141
3238899108,2cyclictest23659-21hostname09:05:151
3238899108,2cyclictest1196-21awk09:25:211
3238899108,1cyclictest9783-21hostname12:00:151
3238899108,1cyclictest25649-21expr09:10:181
3238899108,1cyclictest24614-21sed11:20:251
3238899107,2cyclictest8979-21sed09:40:251
3238899107,2cyclictest12366-21grep09:50:161
3238899103,3cyclictest15265-21grep12:10:181
3238399108,1cyclictest31897-21head10:30:180
3238399108,1cyclictest24522-21sed10:15:140
3238399108,1cyclictest20470-21basename11:15:100
3238399108,1cyclictest20100-21taskset11:13:280
3238399108,1cyclictest17094-21basename12:15:150
3238399108,1cyclictest15595-21cat07:40:170
3238399108,1cyclictest11777-21taskset07:31:430
3238399108,1cyclictest114312sleep008:40:160
3238399107,2cyclictest9275-21grep08:35:160
3238399107,2cyclictest6429-21tr08:30:140
3238399107,2cyclictest3676-21grep10:40:140
3238399107,2cyclictest22998-21cut09:05:120
3238399107,2cyclictest22889-21apache_accesses11:20:110
3238399107,2cyclictest21978-21cat09:00:240
3238399107,2cyclictest21398-21chrt12:22:050
3238399107,2cyclictest19797-21cut07:50:150
3238399107,2cyclictest18791-21sed07:45:260
323939996,2cyclictest15167-21cat07:40:153
323939995,3cyclictest15081-21cut09:55:183
323909997,1cyclictest248742sleep209:08:552
323909994,2cyclictest225-21systemd-journal10:30:002
323889998,1cyclictest10942sleep110:35:041
323839997,1cyclictest15544-21cat12:10:200
323839996,2cyclictest2828-21sed10:35:240
323839991,2cyclictest0-21swapper/007:20:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional