You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-02 - 08:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Fri Jan 02, 2026 00:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
916028876,8sleep00-21swapper/019:07:270
931828370,9sleep30-21swapper/319:09:303
920628265,13sleep10-21swapper/119:08:041
916728269,9sleep20-21swapper/219:07:322
9451992626,0cyclictest0-21swapper/123:00:211
945199241,1cyclictest3493-21df_abs21:10:161
945199240,1cyclictest0-21swapper/119:35:241
945199240,14cyclictest0-21swapper/119:55:141
944999198,6cyclictest11143-21perf20:20:000
9451991813,3cyclictest5244-21ls20:05:201
945199171,2cyclictest0-21swapper/122:00:151
944999173,7cyclictest20622-21munin-run22:55:000
945199164,10cyclictest0-21swapper/120:50:131
944999164,6cyclictest30159-21munin-run21:00:000
944999164,6cyclictest30159-21munin-run21:00:000
944999161,8cyclictest8679-21idleruntime-cro22:30:000
944999161,0cyclictest0-21swapper/000:05:130
945199158,6cyclictest0-21swapper/119:30:131
945199156,6cyclictest1539-21ntp_states23:20:221
944999154,6cyclictest28963-21idleruntime-cro22:05:000
945699145,5cyclictest16859-21unattended-upgr19:24:573
9456991410,3cyclictest30038-21sed00:20:193
9454991410,3cyclictest19665-21tr20:35:262
945499140,1cyclictest0-21swapper/221:05:142
945499140,0cyclictest0-21swapper/223:35:142
9451991412,1cyclictest0-21swapper/123:30:151
9451991410,3cyclictest28254-21tr23:10:151
945199140,2cyclictest0-21swapper/120:20:141
945199140,2cyclictest0-21swapper/100:15:121
944999141,7cyclictest12319-21munin-run21:30:000
9449991411,2cyclictest25453-21wc21:55:210
9449991411,2cyclictest13856-21wc23:45:270
9449991410,3cyclictest8206-21grep20:10:260
9449991410,3cyclictest6552-21wc23:30:260
9449991410,3cyclictest21201-21grep00:00:300
9449991410,3cyclictest12789-21ls22:35:280
944999140,8cyclictest27791-21basename20:55:000
945699138,2cyclictest0-21swapper/322:55:153
9456991310,2cyclictest596-21wc22:10:263
9456991310,2cyclictest3451-21grep00:30:293
9456991310,2cyclictest29034-21awk00:20:013
9456991310,2cyclictest26418-21cat19:45:003
9456991310,2cyclictest26084-21sed21:55:283
9456991310,2cyclictest23598-21cat21:50:273
9456991310,2cyclictest22558-21cut00:05:183
9456991310,2cyclictest21211-21sed00:00:303
9456991310,2cyclictest17042-21cron21:40:003
945499139,3cyclictest25030-21sed19:40:162
945499134,7cyclictest13471-21sh22:40:002
945499131,11cyclictest32133-21users23:15:282
9454991310,2cyclictest30854-21grep19:50:282
9454991310,2cyclictest30700-21grep21:00:182
9454991310,2cyclictest26622-21wc22:00:002
9454991310,2cyclictest23577-21grep20:45:142
9454991310,2cyclictest19245-21head22:50:192
9454991310,2cyclictest1622-21head23:20:232
9454991310,2cyclictest15625-21grep21:35:192
9454991310,2cyclictest14491-21cat19:20:012
9454991310,2cyclictest12150-21head20:20:172
9454991310,2cyclictest1124-21grep00:25:272
945499130,1cyclictest0-21swapper/220:40:532
945199139,3cyclictest11117-21basename22:35:001
945199139,2cyclictest22006-21cut20:40:211
945199137,5cyclictest13325-21expr21:30:161
9451991310,2cyclictest19152-21sort20:35:211
944999139,3cyclictest9021-21wc23:35:270
944999139,3cyclictest3727-21grep21:10:180
944999139,3cyclictest20603-21tail19:30:190
944999139,3cyclictest16209-21wc19:20:250
944999139,2cyclictest30687-21tr23:15:160
944999139,2cyclictest20450-21head21:45:180
944999137,5cyclictest6931-21kworker/u8:022:20:220
944999132,7cyclictest3684-21apt-get22:20:000
9449991310,2cyclictest32091-21grep21:00:320
9449991310,2cyclictest15778-21wc23:50:200
9449991310,2cyclictest12844-21wc20:20:260
945699129,2cyclictest867-21df21:05:163
945699129,2cyclictest6925-21cat21:15:263
945699129,2cyclictest6328-21ls23:30:223
945699129,2cyclictest4852-21grep22:20:163
945699129,2cyclictest3735-21head21:10:193
945699129,2cyclictest3189-21sed22:15:293
945699129,2cyclictest24649-21grep00:10:143
945699129,2cyclictest19939-21grep21:45:163
945699129,2cyclictest1648-21cat20:00:013
945699129,2cyclictest15491-21head23:50:183
945699129,2cyclictest12139-21sed22:35:193
945699128,3cyclictest28273-21grep19:45:293
945699120,11cyclictest6786-21grep20:10:143
945499129,2cyclictest7970-21cat22:25:272
945499129,2cyclictest7315-21sed20:10:182
945499129,2cyclictest6299-21awk22:25:002
945499129,2cyclictest4526-21cat21:10:292
945499129,2cyclictest4390-21grep23:25:322
945499129,2cyclictest3529-21sed00:30:292
945499129,2cyclictest28500-21grep00:15:262
945499129,2cyclictest27577-21grep22:00:172
945499129,2cyclictest27268-21grep20:50:272
945499129,2cyclictest20743-21grep19:30:222
945499129,2cyclictest19043-21sed21:40:282
945499129,2cyclictest1842-21tr22:15:152
945499129,2cyclictest18204-21cut23:55:232
945499129,2cyclictest15286-21cat20:25:292
945499129,2cyclictest1018-21grep19:55:282
945499126,3cyclictest31398-21munin-run00:25:002
945499120,12cyclictest0-21swapper/219:35:142
945199129,2cyclictest7875-21wc22:25:241
945199129,2cyclictest4268-21cut00:35:151
945199129,2cyclictest3906-21cron20:05:001
945199129,2cyclictest30905-21grep00:20:281
945199129,2cyclictest30868-21sed19:50:281
945199129,2cyclictest30023-21tail22:05:221
945199129,2cyclictest29136-21grep20:55:211
945199129,2cyclictest29136-21grep20:55:211
945199129,2cyclictest20988-21grep22:55:161
945199129,2cyclictest18079-21grep21:40:181
945199129,2cyclictest17897-21tail19:25:161
945199129,2cyclictest15464-21grep19:20:161
945199129,2cyclictest14005-21tr20:25:181
945199129,2cyclictest12946-21cut23:45:171
945199129,2cyclictest11462-21wc21:25:241
945199128,3cyclictest11103-21cron20:20:001
945199127,3cyclictest19983-21tr21:45:161
945199121,8cyclictest0-21swapper/100:10:131
945199120,11cyclictest14381-21cut22:40:161
944999129,2cyclictest9326-21cat21:20:260
944999129,2cyclictest7470-21sh21:20:010
944999129,2cyclictest3432-21sed20:00:270
944999129,2cyclictest32573-21grep00:25:170
944999129,2cyclictest28205-21fgrep00:15:220
944999129,2cyclictest25835-21wc00:10:230
944999129,2cyclictest25386-21cron20:50:000
944999129,2cyclictest20983-21grep20:40:140
944999129,2cyclictest18057-21tail21:40:190
944999129,2cyclictest15909-21wc21:35:210
944999129,2cyclictest1545-21awk20:00:000
944999129,2cyclictest15386-21sed22:40:260
944999129,2cyclictest14046-21head19:15:260
944999129,2cyclictest1108-21tail21:05:180
944999128,3cyclictest30013-21grep00:20:200
944999128,3cyclictest26782-21tr23:05:240
944999126,2cyclictest225-21systemd-journal19:50:000
944999124,6cyclictest14078-21munin-node23:02:060
945699118,2cyclictest3981-21sed23:25:253
945699118,2cyclictest32088-21grep23:15:283
945699118,2cyclictest29648-21sed23:10:283
945699118,2cyclictest27633-21chrt20:53:243
945699118,2cyclictest26306-21grep23:05:193
945699118,2cyclictest25387-21cron20:50:003
945699118,2cyclictest25381-21cron23:05:003
945699118,2cyclictest2491-21turbostat.cron23:25:003
945699118,2cyclictest23232-21tr19:35:243
945699118,2cyclictest22613-21grep20:40:293
945699118,2cyclictest2013-21grep20:00:143
945699118,2cyclictest19883-21which20:35:293
945699118,2cyclictest19113-21taskset21:40:503
945699118,2cyclictest18086-21cut19:25:193
945699118,2cyclictest14805-21cat20:25:223
945699118,2cyclictest14122-21seq19:16:083
945699118,2cyclictest10722-21head23:40:173
945699117,3cyclictest13037-21grep20:20:283
945699112,7cyclictest9693-21perl22:30:193
945699111,6cyclictest0-21swapper/323:45:143
945499119,2cyclictest32-21ksoftirqd/219:20:122
945499119,1cyclictest14866-21strings22:40:202
945499118,2cyclictest6935-21taskset23:31:112
945499118,2cyclictest3430-21grep20:00:272
945499118,2cyclictest27753-21cron23:10:002
945499118,2cyclictest26898-21grep19:45:162
945499118,2cyclictest21746-21cat00:05:002
945499118,2cyclictest18268-21sleep20:35:002
945499118,2cyclictest13865-21cat23:45:272
945499118,2cyclictest12327-21basename21:30:012
945499118,2cyclictest11123-21awk20:20:002
945499118,2cyclictest11061-21chrt22:34:402
945499117,3cyclictest5637-21grep21:15:132
945499115,2cyclictest0-21swapper/221:30:122
945199118,2cyclictest8263-21grep20:10:271
945199118,2cyclictest7615-21tr23:35:151
945199118,2cyclictest7441-21taskset21:19:471
945199118,2cyclictest4854-21sort22:20:161
945199118,2cyclictest26931-21cut19:45:161
945199118,2cyclictest25929-21tr23:05:171
945199118,2cyclictest24736-21tr21:55:151
945199118,2cyclictest24570-21tr19:40:141
945199118,2cyclictest18784-21tr22:50:161
945199118,2cyclictest17764-21grep23:55:201
945199118,2cyclictest16911-21head20:30:201
945199118,2cyclictest15019-21grep21:35:161
945199118,2cyclictest14168-21taskset19:16:421
945199118,2cyclictest13089-21head22:35:291
945199118,2cyclictest11624-21sed19:10:281
945199118,2cyclictest1146-21head00:25:271
945199117,3cyclictest31691-21wc21:00:271
945199117,3cyclictest16818-21cron23:55:001
945199114,5cyclictest455-21dbus-daemon00:08:121
944999118,2cyclictest9957-21wc22:30:210
944999118,2cyclictest5798-21expr00:35:290
944999118,2cyclictest4681-21chrt23:27:460
944999118,2cyclictest28167-21grep23:10:150
944999118,2cyclictest22361-21tr21:50:150
944999118,2cyclictest21623-21grep22:55:190
944999118,2cyclictest17745-21sed22:45:280
944999118,2cyclictest16879-21tail20:30:200
944999118,2cyclictest12053-21sh19:15:000
944999117,3cyclictest17705-21taskset19:25:150
944999110,10cyclictest30566-21find19:50:240
945699108,1cyclictest8897-21taskset20:15:083
945699108,1cyclictest4850-21tune2fs20:05:173
945699108,1cyclictest29491-21grep19:50:163
945699108,1cyclictest29374-21wget22:05:183
945699108,1cyclictest28181-21grep20:55:153
945699108,1cyclictest28181-21grep20:55:153
945699108,1cyclictest27219-21expr22:00:153
945699108,1cyclictest21399-21taskset19:32:163
945699108,1cyclictest188532sleep323:55:423
945699108,1cyclictest181232sleep320:33:273
945699108,1cyclictest17530-21sensors22:45:263
945699108,1cyclictest13716-21seq22:40:023
945699108,1cyclictest13580-21cat21:30:193
945699108,1cyclictest10620-21cat21:25:163
945699107,2cyclictest8468-21head21:20:163
945699107,2cyclictest7649-21tr23:35:153
945699107,2cyclictest7277-21grep22:25:183
945699107,2cyclictest4155-21seq00:35:053
945699107,2cyclictest20216-21sed22:50:293
945699106,3cyclictest32005-21grep00:25:143
945699100,9cyclictest9903-21seq19:10:023
945699100,9cyclictest30863-21expr21:00:183
945499108,1cyclictest5158-21wc00:35:212
945499108,1cyclictest17141-21apache_volume19:25:122
945499108,1cyclictest14753-21wget23:50:132
945499107,2cyclictest8023-21tr21:20:142
945499107,2cyclictest449-21irqbalance22:57:412
945499107,2cyclictest32413-21hostname22:10:182
945499107,2cyclictest29232-21apache_accesses22:05:162
945499107,2cyclictest28072-21cut20:55:142
945499107,2cyclictest28072-21cut20:55:142
945499107,2cyclictest25375-21taskset23:04:582
945499107,2cyclictest24757-21grep00:10:142
945499107,2cyclictest23283-21ls21:50:232
945499107,2cyclictest20500-21sed21:45:182
945499107,2cyclictest16905-21grep22:45:192
945499107,2cyclictest11539-21cat19:10:272
945499107,2cyclictest10005-21wget23:40:132
945499100,9cyclictest41912sleep220:05:132
945499100,9cyclictest28031-21apache_volume23:10:142
945199108,1cyclictest45462sleep123:26:031
945199108,1cyclictest22312-21grep21:50:151
945199108,1cyclictest17885-21taskset22:45:341
945199108,1cyclictest0-21swapper/122:15:151
945199107,2cyclictest8863-21head21:20:201
945199107,2cyclictest1964-21cut00:30:151
945199107,2cyclictest11720-21taskset23:40:331
945199107,2cyclictest1047-21grep21:05:181
945199106,3cyclictest20713-21tr00:00:251
945199105,4cyclictest31543-21grep23:15:221
945199103,3cyclictest225-21systemd-journal20:50:001
945199100,9cyclictest32647-21sed22:10:201
944999108,1cyclictest31675-21wget22:10:140
944999108,1cyclictest30040-21sed22:05:210
944999108,1cyclictest2461-21taskset23:24:460
944999108,1cyclictest1903-21aten2_r7power_p00:30:150
944999108,1cyclictest14462-21cut20:25:200
944999108,1cyclictest12985-21cat21:30:140
944999107,2cyclictest19744-21sed20:35:270
944999107,2cyclictest18194-21sed23:55:230
944999100,9cyclictest26055-21taskset19:40:500
94549998,1cyclictest22248-21expr00:05:162
94499997,1cyclictest11750-21chrt23:40:570
94499996,2cyclictest4182-21apache_processe20:05:120
94499994,3cyclictest0-21swapper/019:35:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional