You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-13 - 09:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Wed May 13, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1380329961,33sleep00-21swapper/019:07:500
1396129777,16sleep30-21swapper/319:09:543
1371629280,8sleep20-21swapper/219:06:442
1395928568,13sleep10-21swapper/119:09:521
1406399290,4cyclictest141rcu_preempt23:00:173
14054992727,0cyclictest0-21swapper/000:11:380
14060992421,2cyclictest32-21ksoftirqd/200:30:002
1406099210,0cyclictest0-21swapper/221:05:362
1405899210,21cyclictest0-21swapper/121:32:401
1405899210,21cyclictest0-21swapper/121:28:411
1405899210,21cyclictest0-21swapper/100:38:381
1406099200,1cyclictest0-21swapper/200:05:172
1405899201,18cyclictest0-21swapper/100:15:141
1405899200,20cyclictest0-21swapper/122:52:581
1405899200,1cyclictest0-21swapper/120:05:241
14063991916,2cyclictest39-21ksoftirqd/322:56:283
1406099190,19cyclictest0-21swapper/223:35:002
14058991914,4cyclictest0-21swapper/121:44:561
14058991914,4cyclictest0-21swapper/121:44:551
1405899190,19cyclictest0-21swapper/123:40:141
1405499190,1cyclictest0-21swapper/022:23:170
1406099180,18cyclictest0-21swapper/223:10:322
1405899184,11cyclictest14405-21runrttasks21:53:011
1405899180,18cyclictest0-21swapper/121:48:021
14054991817,1cyclictest0-21swapper/023:40:230
14054991816,2cyclictest13-21ksoftirqd/000:36:500
1405499180,1cyclictest0-21swapper/023:38:560
14063991715,2cyclictest94-21kswapd021:39:013
14060991717,0cyclictest0-21swapper/222:57:142
1406099170,16cyclictest0-21swapper/223:44:312
1405899170,16cyclictest0-21swapper/123:22:401
1405499170,16cyclictest0-21swapper/023:28:270
1405499170,16cyclictest0-21swapper/022:59:260
1406399163,12cyclictest1-21systemd00:08:403
1406099169,6cyclictest1035-21packagekitd21:11:542
14060991615,1cyclictest0-21swapper/223:04:092
1406099160,0cyclictest0-21swapper/222:29:342
1405899169,5cyclictest38150irq/131-snd_hda23:10:001
14058991613,2cyclictest25-21ksoftirqd/121:11:251
1405899161,1cyclictest6651-21diskmemload23:13:331
1405499162,3cyclictest141rcu_preempt22:40:070
1405499161,12cyclictest0-21swapper/022:45:140
1405499160,3cyclictest0-21swapper/021:55:120
1405499160,1cyclictest0-21swapper/023:04:010
1405499160,1cyclictest0-21swapper/022:33:440
1405499160,12cyclictest0-21swapper/019:35:130
1406399153,7cyclictest225-21systemd-journal23:18:393
1406399153,7cyclictest225-21systemd-journal23:18:393
1406399150,1cyclictest0-21swapper/323:28:273
1406099150,15cyclictest0-21swapper/222:23:172
14058991514,1cyclictest0-21swapper/122:23:311
1405899150,14cyclictest12463-21kworker/1:221:58:011
1405499153,6cyclictest22119-21apt-get20:35:000
14054991510,3cyclictest537-21snmpd23:54:390
1405499150,13cyclictest0-21swapper/000:22:480
1406399144,5cyclictest11462-21perf22:50:003
1406399141,1cyclictest0-21swapper/322:54:163
1406399141,1cyclictest0-21swapper/321:16:393
14063991411,2cyclictest1200-21sh22:04:593
1406399140,14cyclictest0-21swapper/321:27:383
1406099142,6cyclictest225-21systemd-journal23:20:002
1406099142,6cyclictest225-21systemd-journal23:20:002
14060991411,2cyclictest23540-21sh00:15:002
14060991410,3cyclictest25245-21grep00:35:152
1405899149,4cyclictest28850-21sh23:20:001
1405899149,4cyclictest28850-21sh23:19:591
14058991413,1cyclictest0-21swapper/122:40:181
14058991410,2cyclictest19251-21grep20:25:271
1405899140,13cyclictest0-21swapper/123:47:151
1405899140,12cyclictest0-21swapper/122:31:521
1405499143,3cyclictest225-21systemd-journal19:33:380
1405499142,6cyclictest15017-21awk20:20:010
1405499140,1cyclictest0-21swapper/000:01:590
1405499140,13cyclictest0-21swapper/022:29:180
1406399139,3cyclictest32207-21ls19:45:203
1406399139,3cyclictest32207-21ls19:45:203
1406399139,3cyclictest23340-21tr20:35:163
14063991313,0cyclictest0-21swapper/300:27:333
14063991310,2cyclictest24500-21wc23:55:243
14063991310,2cyclictest17685-21sed00:10:293
14063991310,2cyclictest14249-21cat23:50:013
1406099139,3cyclictest17051-21expr00:30:152
1406099137,4cyclictest3952-21cut22:45:152
1406099133,8cyclictest11192-21/usr/sbin/munin22:30:132
14060991312,1cyclictest0-21swapper/222:07:022
14060991310,2cyclictest31688-21grep19:45:162
14060991310,2cyclictest31688-21grep19:45:162
14060991310,2cyclictest26821-21grep21:40:252
14060991310,2cyclictest26821-21grep21:40:252
14060991310,2cyclictest18099-21forks21:55:172
14060991310,2cyclictest1233-21cat21:25:242
1406099130,12cyclictest0-21swapper/222:11:172
1406099130,0cyclictest0-21swapper/200:16:102
1405899139,3cyclictest4508-21cut22:45:181
1405899138,4cyclictest2938-21taskset19:53:211
1405899138,4cyclictest11374-21awk22:10:211
1405899138,3cyclictest14348-21basename19:10:141
1405899133,9cyclictest17338-21perl21:35:171
14058991310,2cyclictest9178-21head00:25:181
14058991310,2cyclictest5014-21sed19:55:261
14058991310,2cyclictest3367-21grep22:25:171
14058991310,2cyclictest2185-21grep21:00:151
14058991310,2cyclictest1876-21idleruntime-cro21:00:001
1405899130,12cyclictest9512-21basename22:10:011
1405499139,3cyclictest32122-21uname19:45:190
1405499139,3cyclictest32122-21uname19:45:190
1405499139,3cyclictest28871-21turbostat.cron23:20:000
1405499139,3cyclictest28871-21turbostat.cron23:20:000
1405499139,3cyclictest25798-21ssh00:15:260
1405499139,2cyclictest1385-21cat22:05:010
1405499133,6cyclictest7199-21aten2_r7power_c00:05:140
14054991313,0cyclictest0-21swapper/023:47:320
1405499131,2cyclictest1686-21apt-get22:05:120
14054991310,2cyclictest26754-21grep21:40:250
14054991310,2cyclictest26754-21grep21:40:250
14054991310,2cyclictest11088-21sort20:10:180
1405499130,3cyclictest0-21swapper/023:07:170
1405499130,12cyclictest6383-21grep20:00:160
1405499130,12cyclictest0-21swapper/021:33:410
1406399129,2cyclictest32132-21ssh23:40:253
1406399129,2cyclictest31932-21ssh22:23:003
1406399129,2cyclictest31715-21cat21:25:003
1406399129,2cyclictest25231-21tr20:40:163
1406399129,2cyclictest23966-21grep23:35:273
1406399129,2cyclictest10373-21cut21:50:203
1406399124,4cyclictest452-21systemd-logind23:09:043
14063991211,1cyclictest0-21swapper/321:43:013
14063991211,1cyclictest0-21swapper/321:43:013
14063991210,1cyclictest17395-21head19:15:183
1406399120,11cyclictest26323-21ssh00:15:393
1406399120,10cyclictest6641-21sh00:05:003
1406099129,2cyclictest6651-21diskmemload22:43:542
1406099129,2cyclictest24699-21grep23:55:252
1406099129,2cyclictest20661-21wc22:35:222
1406099129,2cyclictest18320-21grep19:15:272
1406099129,2cyclictest17384-21turbostat.cron20:25:012
1406099129,2cyclictest16521-21grep23:50:272
1406099129,2cyclictest1556-21grep21:45:172
1406099128,4cyclictest32-21ksoftirqd/219:25:142
1406099128,3cyclictest2396-21grep21:00:152
1406099128,3cyclictest12099-21grep20:10:262
1406099128,3cyclictest0-21swapper/222:16:262
1406099127,2cyclictest0-21swapper/220:55:132
1405899129,2cyclictest6506-21cut20:00:161
1405899129,2cyclictest6201-21cut23:25:181
1405899129,2cyclictest28097-21grep20:45:171
1405899129,2cyclictest25640-21cut20:40:181
1405899129,2cyclictest23831-21head20:35:201
1405899129,2cyclictest21001-21grep22:55:181
1405899129,2cyclictest16937-21grep21:15:231
1405899128,3cyclictest32672-21sort00:20:141
14058991211,1cyclictest26882-21kworker/1:000:02:231
14058991210,1cyclictest0-21swapper/123:57:281
14058991210,1cyclictest0-21swapper/123:50:471
1405899120,12cyclictest0-21swapper/123:39:291
1405499129,2cyclictest7701-21cat21:30:000
1405499129,2cyclictest4577-21grep19:55:210
1405499129,2cyclictest30286-21grep19:40:240
1405499129,2cyclictest29061-21chrt23:58:450
1405499129,2cyclictest23301-21grep20:35:160
1405499129,2cyclictest1570-21sort19:50:170
1405499129,2cyclictest15463-21wc23:30:250
1405499129,2cyclictest13633-21sed23:10:190
1405499128,3cyclictest731-21wc20:55:190
1405499128,2cyclictest18985-21wc19:20:000
1405499128,2cyclictest17439-21ssh21:35:170
1405499120,11cyclictest12818-21grep22:50:170
134782120,0chrt0-21swapper/021:52:180
257492110,0chrt0-21swapper/122:00:131
1406399119,1cyclictest25127-21cut00:35:153
1406399118,2cyclictest9870-21chrt20:05:393
1406399118,2cyclictest6882-21ssh21:48:383
1406399118,2cyclictest6177-21expr20:00:143
1406399118,2cyclictest5172-21grep21:05:163
1406399118,2cyclictest4624-21ssh22:25:283
1406399118,2cyclictest3084-21ssh00:21:183
1406399118,2cyclictest30192-21sed19:40:243
1406399118,2cyclictest30172-21grep23:20:163
1406399118,2cyclictest27698-21basename22:40:123
1406399118,2cyclictest24552-21head19:30:173
1406399118,2cyclictest23246-21taskset19:26:213
1406399118,2cyclictest21974-21ssh22:36:033
1406399118,2cyclictest19849-21ssh23:53:023
1406399118,2cyclictest19206-21ssh00:30:513
1406399118,2cyclictest18348-21cut22:15:133
1406399118,2cyclictest17418-21ssh23:12:333
1406399118,2cyclictest17244-21taskset21:55:053
1406399118,2cyclictest15697-21ssh23:30:283
1406399118,2cyclictest15517-21tr20:20:143
1406399118,2cyclictest15419-21awk19:10:203
1406399118,2cyclictest13613-21ssh22:31:003
1406399118,2cyclictest12833-21ssh21:13:203
1406399118,2cyclictest10240-21tr22:10:143
1406399110,10cyclictest14032-21ssh21:33:243
1406399110,10cyclictest10698-21cat20:10:163
1406099119,1cyclictest27504-21grep20:45:132
1406099118,2cyclictest8684-21ssh23:26:302
1406099118,2cyclictest6080-21cat23:05:252
1406099118,2cyclictest4570-21ssh00:22:292
1406099118,2cyclictest31216-21sh00:00:132
1406099118,2cyclictest30879-21tr20:50:212
1406099118,2cyclictest29291-21seq23:20:102
1406099118,2cyclictest28218-21ssh21:22:232
1406099118,2cyclictest24920-21chrt19:30:202
1406099118,2cyclictest24914-21cat20:40:002
1406099118,2cyclictest21151-21chrt19:24:552
1406099118,2cyclictest18078-21fgrep21:35:222
1406099118,2cyclictest17693-21grep20:25:142
1406099118,2cyclictest15727-21cat21:15:152
1406099118,2cyclictest12627-21sh21:32:192
1406099118,2cyclictest12056-21cut22:50:142
1406099118,2cyclictest10998-21ssh23:47:382
1406099117,3cyclictest9114-21ssh21:50:112
1406099111,9cyclictest26074-21grep22:00:152
1406099110,10cyclictest26188-21apache_volume19:35:122
1405899118,2cyclictest31675-21taskset20:52:101
1405899118,2cyclictest30437-21tail23:00:291
1405899118,2cyclictest29291-21grep19:40:141
1405899118,2cyclictest26415-21tr19:35:141
1405899118,2cyclictest22634-21cut19:25:221
1405899118,2cyclictest20267-21tr20:30:131
1405899118,2cyclictest19794-21grep19:20:171
1405899118,2cyclictest18481-21rm23:32:391
1405899118,2cyclictest18292-21aten2_r7power_c22:15:131
1405899118,2cyclictest18035-21ssh00:30:231
1405899118,2cyclictest17317-21cut19:15:171
1405899118,2cyclictest15589-21cat20:20:141
1405899118,2cyclictest12894-21wget20:15:141
1405899117,3cyclictest32103-21cut19:45:191
1405899117,3cyclictest32103-21cut19:45:191
1405899117,3cyclictest27026-21http22:40:001
1405899115,3cyclictest225-21systemd-journal00:08:411
1405899110,6cyclictest25314-21sensors21:20:211
1405899110,10cyclictest16052-21cut00:10:171
1405499118,2cyclictest8241-21ssh21:10:210
1405499118,2cyclictest6661-21basename21:10:000
1405499118,2cyclictest3167-21ssh21:45:410
1405499118,2cyclictest20967-21ssh22:15:580
1405499118,2cyclictest20028-21grep22:35:170
1405499118,2cyclictest18867-21grep20:25:220
1405499118,2cyclictest18748-21ssh21:16:280
1405499118,2cyclictest16567-21ssh00:30:100
1405499118,2cyclictest14936-21cut19:10:160
1405499118,2cyclictest11737-21chrt00:26:340
1405499118,2cyclictest10843-21grep22:10:170
1405499110,10cyclictest25885-21ssh21:20:330
1405499110,0cyclictest0-21swapper/023:21:070
1406399108,1cyclictest291682sleep320:46:143
1406399108,1cyclictest20854-21taskset19:21:043
1406399108,1cyclictest1588-21ssh22:05:093
1406399108,1cyclictest12929-21grep20:15:143
1406399107,2cyclictest32131-21chrt20:55:023
1406399107,2cyclictest31003-21uname20:50:223
1406399107,2cyclictest27787-21chrt19:35:253
1406399107,2cyclictest1927-21cat19:50:193
1406399107,2cyclictest17787-21wc20:25:153
1406399106,3cyclictest20772-21head20:30:153
1406399100,9cyclictest4386-21cat19:55:193
1406099108,1cyclictest6964-21grep20:00:212
1406099108,1cyclictest4177-21basename19:55:172
1406099108,1cyclictest28929-21expr19:40:112
1406099108,1cyclictest14558-21grep19:10:142
1406099107,2cyclictest25367-21cat20:40:162
1406099107,2cyclictest21815-21ssh23:35:052
1406099107,2cyclictest20146-21cut20:30:122
1406099107,2cyclictest14067-21cut20:15:222
1406099107,2cyclictest1098-21wget19:50:142
1406099107,2cyclictest10023-21taskset20:07:372
1405899107,2cyclictest5581-21grep21:05:181
1405899107,2cyclictest10775-21grep20:10:161
1405899101,7cyclictest0-21swapper/119:30:131
1405499108,1cyclictest296972sleep020:50:030
1405499108,1cyclictest25669-21fschecks_time20:40:180
1405499107,2cyclictest21436-21apache_processe19:25:130
1405499107,2cyclictest19436-21expr19:20:150
1405499107,2cyclictest16034-21tail20:20:170
1405499100,9cyclictest3960-21taskset21:01:180
140639997,1cyclictest3302-21ntp_states21:00:213
140549997,1cyclictest9685-21cat20:05:250
140549997,1cyclictest291352sleep020:45:460
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional