You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-04 - 01:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Fri Apr 03, 2026 12:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15622995429,11cyclictest26008-21apt-get12:25:142
15622995112,14cyclictest27-21ksoftirqd/209:45:132
1554999498,6cyclictest131rcu_preempt10:15:120
1554999461,39cyclictest4959-21ssh09:30:140
15622994224,14cyclictest25351-21apt-get11:30:132
1558399423,36cyclictest21863-21rm11:10:121
1554999422,30cyclictest16367-21ssh11:25:130
15622994122,15cyclictest0-21swapper/207:55:112
1554999413,29cyclictest5028-21ssh11:55:130
15633994026,8cyclictest28366irq/58-eth011:50:123
1563399402,24cyclictest28366irq/58-eth008:15:103
1554999408,6cyclictest131rcu_preempt09:40:120
15633993921,11cyclictest28366irq/58-eth010:25:113
1562299398,8cyclictest14226-21ssh09:15:252
1562299396,14cyclictest23946-21cat11:45:292
15622993929,8cyclictest8152-21apt-get12:15:132
15622993915,22cyclictest28604-21ssh11:50:142
1562299389,21cyclictest16954-21cat11:05:292
1562299387,15cyclictest8820-21mailstats11:55:272
15622993810,25cyclictest0-21swapper/209:35:142
15583993810,6cyclictest131rcu_preempt09:15:121
1554999382,27cyclictest19537-21ssh09:20:120
1554999382,27cyclictest19537-21ssh09:20:120
1562299378,19cyclictest6765-21usage_EthcTask11:35:312
1562299377,15cyclictest23118-21mailstats08:55:262
15622993724,8cyclictest22789-21apt-get09:40:122
15622993721,11cyclictest28455-21ssh09:25:112
15622993712,5cyclictest26783-21find12:05:292
1562299367,19cyclictest5887-21cat09:10:302
15622993628,3cyclictest15755-21apt-get07:10:122
15622993618,13cyclictest8300-21ssh09:50:142
1554999363,18cyclictest525-21ssh10:40:110
1554999362,9cyclictest12-21ksoftirqd/010:10:130
1554999362,2cyclictest10612-21ssh09:15:130
1562299358,18cyclictest13784-21ssh10:28:582
1562299358,13cyclictest25970-21irqrtprio07:20:172
1562299358,13cyclictest25970-21irqrtprio07:20:172
15622993510,14cyclictest315441idleruntime08:00:152
1554999357,5cyclictest131rcu_preempt10:35:140
1563399344,16cyclictest28366irq/58-eth011:35:123
1562299349,9cyclictest13385-21latency_hist08:15:012
1562299349,17cyclictest27139-21cat08:25:282
1562299349,17cyclictest27139-21cat08:25:282
1562299347,18cyclictest0-21swapper/210:15:072
1562299347,17cyclictest1043-21ssh10:03:232
1562299347,16cyclictest909-21ntp_states10:20:262
1562299346,7cyclictest0-21swapper/211:10:322
1562299346,12cyclictest11090-21ntp_states11:20:262
1562299346,10cyclictest6792-21grep09:30:212
1562299345,20cyclictest10916-21ssh11:40:162
1562299344,3cyclictest131rcu_preempt10:35:142
15622993413,17cyclictest0-21swapper/210:55:122
1563399331,20cyclictest28366irq/58-eth008:25:123
1563399331,20cyclictest28366irq/58-eth008:25:123
1562299338,8cyclictest5960-21awk07:30:232
1562299338,7cyclictest1306-21diskmemload10:12:142
1562299336,17cyclictest1313-21/usr/sbin/munin09:10:012
1562299336,14cyclictest32501-21latency_hist08:35:012
1562299335,19cyclictest0-21swapper/209:57:522
1562299335,14cyclictest7240-21cut08:40:202
1562299335,13cyclictest18774-21ssh12:38:352
15622993325,5cyclictest17084-21apt-get12:20:122
15622993325,5cyclictest17084-21apt-get12:20:122
1558399338,3cyclictest0-21swapper/109:10:121
1554999333,23cyclictest18507-21rm10:50:120
1554999332,21cyclictest9587-21ssh10:45:140
1563399321,16cyclictest28366irq/58-eth010:15:133
1562299328,14cyclictest5447-21users10:40:302
1562299325,9cyclictest28221-21date09:25:012
1562299325,9cyclictest28221-21date09:25:012
15622993227,2cyclictest2442-21apt-get12:30:122
15622993217,11cyclictest30770-21apt-get11:15:112
15622993214,8cyclictest18065-21ntp_192.168.11408:50:242
15622993213,10cyclictest22269-21ssh11:26:422
1558399324,3cyclictest131rcu_preempt07:10:121
1558399321,28cyclictest31754-21ssh09:45:121
1558399321,22cyclictest16386-21ssh11:25:141
15549993218,7cyclictest0-21swapper/012:35:120
1563399315,17cyclictest28366irq/58-eth009:50:133
1562299315,7cyclictest7283-21usage_Anybus10:05:312
1562299315,12cyclictest18497-21apt-get10:50:122
1562299313,14cyclictest7900-21cut07:35:182
15622993115,12cyclictest31681-21ssh12:10:122
1558399312,8cyclictest21-21ksoftirqd/111:45:121
1558399311,4cyclictest131rcu_preempt10:55:111
1558399311,25cyclictest10918-21rm11:40:161
1562299306,11cyclictest25001-21irqstats09:00:182
1562299304,12cyclictest17154-21cat08:15:262
1558399302,15cyclictest22787-21ssh09:40:111
1558399301,26cyclictest12914-21rm11:05:121
1554999303,21cyclictest31710-21ssh12:10:140
1563399292,18cyclictest28366irq/58-eth011:55:123
1562299298,8cyclictest14025-21cut07:40:202
1562299297,7cyclictest18926-21irqstats08:20:172
1562299296,9cyclictest6605-21kthreadcore11:00:222
1562299296,8cyclictest31516-21cut07:25:182
1562299295,7cyclictest0-21swapper/207:50:242
1562299295,10cyclictest17073-21kthreadcore10:30:192
15622992922,4cyclictest9573-21apt-get10:45:142
1562299291,17cyclictest13778-21ssh12:00:062
1558399298,3cyclictest131rcu_preempt08:15:111
1558399291,20cyclictest26014-21sh12:25:131
1554999296,19cyclictest12-21ksoftirqd/012:20:120
1554999296,19cyclictest12-21ksoftirqd/012:20:120
1554999291,7cyclictest12-21ksoftirqd/011:15:120
15549992912,8cyclictest0-21swapper/012:08:000
15549992911,12cyclictest12-21ksoftirqd/011:45:130
15549992910,15cyclictest131rcu_preempt09:45:110
1563399287,10cyclictest28366irq/58-eth010:55:123
15622992814,11cyclictest924-21snmpd08:37:412
1558399281,23cyclictest3945-21ssh11:00:131
1554999288,16cyclictest12-21ksoftirqd/010:00:110
1554999282,20cyclictest4317-21ssh11:54:380
1563399273,15cyclictest28366irq/58-eth010:50:123
1563399273,15cyclictest0-21swapper/309:22:443
1563399273,15cyclictest0-21swapper/309:22:443
1563399273,14cyclictest28366irq/58-eth011:15:113
15633992715,7cyclictest28366irq/58-eth009:35:143
1562299276,7cyclictest0-21swapper/208:45:202
1562299274,7cyclictest21786-21kthreadcore07:15:192
15622992715,7cyclictest0-21swapper/207:45:112
1554999272,15cyclictest13879-21ssh12:00:120
15549992717,7cyclictest5741-21cron10:25:010
1563399265,12cyclictest0-21swapper/309:30:143
1563399264,15cyclictest0-21swapper/311:40:163
1563399263,16cyclictest0-21swapper/310:02:443
1563399263,15cyclictest0-21swapper/312:18:473
1563399262,17cyclictest0-21swapper/310:39:303
1558399266,4cyclictest131rcu_preempt09:05:131
1558399265,3cyclictest131rcu_preempt08:35:111
1554999262,19cyclictest2849-21ssh10:05:160
1563399253,14cyclictest28366irq/58-eth012:35:123
1563399251,14cyclictest28366irq/58-eth007:25:103
1562299258,14cyclictest8277-21sh08:10:012
1558399259,7cyclictest21-21ksoftirqd/112:13:301
1558399257,5cyclictest21-21ksoftirqd/110:44:211
15583992513,3cyclictest0-21swapper/108:40:121
1554999253,3cyclictest131rcu_preempt10:25:120
1563399243,14cyclictest0-21swapper/310:45:143
1563399243,14cyclictest0-21swapper/309:19:193
1563399242,15cyclictest28366irq/58-eth012:05:143
1558399242,6cyclictest21-21ksoftirqd/111:20:131
15583992411,3cyclictest131rcu_preempt12:35:251
1558399240,6cyclictest73-21kswapd010:00:231
1554999247,12cyclictest151rcuc/008:20:130
1554999244,5cyclictest1295-21awk07:25:250
1554999243,14cyclictest8183-21ssh12:15:140
1554999243,14cyclictest7512-21kworker/u8:312:25:130
1563399236,11cyclictest0-21swapper/309:44:443
1563399234,13cyclictest28366irq/58-eth011:00:133
1563399232,13cyclictest28366irq/58-eth010:10:153
1558399239,8cyclictest1306-21diskmemload11:32:251
1558399238,3cyclictest131rcu_preempt10:35:271
1558399237,5cyclictest21-21ksoftirqd/110:49:311
1558399237,12cyclictest131rcu_preempt12:15:241
1558399235,4cyclictest131rcu_preempt10:20:291
1558399235,3cyclictest131rcu_preempt07:20:131
1558399235,3cyclictest131rcu_preempt07:20:131
1558399235,11cyclictest131rcu_preempt11:20:021
1558399232,16cyclictest201rcuc/108:50:131
1558399231,4cyclictest0-21swapper/107:50:281
1558399231,18cyclictest15079-21sh10:30:131
1558399231,17cyclictest12000-21ls07:40:021
15583992311,3cyclictest21-21ksoftirqd/109:55:321
15583992311,3cyclictest21-21ksoftirqd/109:30:211
15583992310,7cyclictest21-21ksoftirqd/107:30:001
1554999235,4cyclictest22162-21ssh09:39:440
1554999232,16cyclictest7400-21ssh11:20:120
1554999231,4cyclictest0-21swapper/011:07:580
15549992312,3cyclictest0-21swapper/011:43:150
1563399222,13cyclictest28366irq/58-eth009:45:123
1563399221,11cyclictest28366irq/58-eth008:30:123
1558399229,4cyclictest21-21ksoftirqd/111:56:211
1558399229,3cyclictest131rcu_preempt12:20:301
1558399229,3cyclictest131rcu_preempt12:20:301
1558399226,10cyclictest21-21ksoftirqd/109:25:121
1558399225,8cyclictest21-21ksoftirqd/109:50:121
1558399225,13cyclictest6144-21ssh10:25:131
1558399223,9cyclictest131rcu_preempt11:35:171
1558399221,3cyclictest131rcu_preempt09:35:121
15583992213,6cyclictest23832-21memory07:15:211
15583992211,3cyclictest1306-21diskmemload10:08:261
15583992210,3cyclictest21-21ksoftirqd/112:05:131
1563399213,12cyclictest0-21swapper/309:25:123
1563399211,15cyclictest0-21swapper/308:20:153
1558399216,8cyclictest21-21ksoftirqd/107:55:171
1558399216,4cyclictest21-21ksoftirqd/107:50:011
1558399215,8cyclictest21-21ksoftirqd/112:03:261
1558399215,3cyclictest131rcu_preempt08:30:171
1558399212,17cyclictest1-21systemd10:14:271
15583992113,5cyclictest6507-21unixbench_multi12:30:281
1558399211,2cyclictest131rcu_preempt09:20:131
1558399211,2cyclictest131rcu_preempt09:20:131
1558399211,15cyclictest21983-21ls08:20:271
1558399211,14cyclictest3799-21ls08:05:131
1558399211,13cyclictest131rcu_preempt09:00:181
1558399210,15cyclictest20802-21ls10:15:131
1554999219,3cyclictest0-21swapper/011:02:550
1554999218,9cyclictest1306-21diskmemload12:30:120
1554999212,9cyclictest12-21ksoftirqd/007:10:120
1554999212,2cyclictest131rcu_preempt10:55:140
1554999211,5cyclictest0-21swapper/011:32:230
1554999211,3cyclictest131rcu_preempt11:38:520
1554999211,3cyclictest131rcu_preempt07:20:250
1554999211,3cyclictest131rcu_preempt07:20:250
1563399209,6cyclictest0-21swapper/312:30:073
1563399209,6cyclictest0-21swapper/310:30:403
1558399207,10cyclictest7093-21cron07:35:011
1558399206,4cyclictest21-21ksoftirqd/110:52:441
1558399205,3cyclictest131rcu_preempt08:00:191
15583992010,6cyclictest131rcu_preempt08:25:131
15583992010,6cyclictest131rcu_preempt08:25:131
1554999203,15cyclictest9727-21kthreadcore08:10:200
1554999202,2cyclictest131rcu_preempt09:28:050
1554999201,16cyclictest19784-21ls08:55:150
1554999200,3cyclictest131rcu_preempt09:56:300
1563399199,7cyclictest28366irq/58-eth011:05:113
1563399193,11cyclictest0-21swapper/310:07:443
1563399192,9cyclictest0-21swapper/308:00:123
1563399192,11cyclictest0-21swapper/308:50:133
1563399192,11cyclictest0-21swapper/308:10:143
1563399192,10cyclictest0-21swapper/308:55:043
1563399191,5cyclictest28466irq/59-eth010:40:113
15633991913,4cyclictest0-21swapper/311:30:123
15633991912,4cyclictest0-21swapper/312:25:213
15633991912,4cyclictest0-21swapper/310:20:313
15633991910,6cyclictest0-21swapper/312:20:493
15633991910,6cyclictest0-21swapper/312:20:483
1558399196,5cyclictest21-21ksoftirqd/111:50:121
1554999199,8cyclictest32685-21apt-get08:35:130
1554999193,6cyclictest16279-21kthreadcore08:15:200
1554999191,8cyclictest131rcu_preempt09:50:140
1554999191,3cyclictest131rcu_preempt10:30:270
1554999191,3cyclictest131rcu_preempt07:40:160
15549991913,4cyclictest0-21swapper/011:10:130
1554999191,14cyclictest5065-21apt-get08:40:120
1563399188,8cyclictest0-21swapper/312:10:293
1563399185,10cyclictest0-21swapper/311:45:283
1563399182,11cyclictest0-21swapper/309:10:143
1563399182,10cyclictest0-21swapper/311:25:043
1563399182,10cyclictest0-21swapper/309:05:133
1563399182,10cyclictest0-21swapper/308:40:133
1563399182,10cyclictest0-21swapper/307:50:123
1563399182,10cyclictest0-21swapper/307:35:133
1563399182,10cyclictest0-21swapper/307:30:123
1563399182,10cyclictest0-21swapper/307:20:133
1563399182,10cyclictest0-21swapper/307:20:133
1563399182,10cyclictest0-21swapper/307:15:123
1563399181,11cyclictest28366irq/58-eth012:00:123
15633991810,6cyclictest0-21swapper/311:14:263
1558399185,10cyclictest8655-21fschecks_count08:10:171
1558399184,3cyclictest131rcu_preempt07:10:021
1558399181,4cyclictest21-21ksoftirqd/107:45:011
1554999184,3cyclictest0-21swapper/009:00:210
1554999182,2cyclictest131rcu_preempt08:50:220
1554999182,2cyclictest131rcu_preempt07:30:230
15549991813,3cyclictest0-21swapper/009:10:180
1554999181,2cyclictest131rcu_preempt07:50:170
15549991812,4cyclictest0-21swapper/009:05:190
15549991812,4cyclictest0-21swapper/007:15:130
1563399179,6cyclictest0-21swapper/311:20:373
1563399172,10cyclictest0-21swapper/309:55:143
1563399172,10cyclictest0-21swapper/309:00:143
1563399172,10cyclictest0-21swapper/308:35:133
1563399172,10cyclictest0-21swapper/308:05:133
1563399172,10cyclictest0-21swapper/307:55:133
1563399172,10cyclictest0-21swapper/307:45:133
1563399172,10cyclictest0-21swapper/307:40:143
1562299171,14cyclictest15524-21cyclictest07:10:012
15583991711,4cyclictest0-21swapper/108:55:111
1558399170,5cyclictest9854-21/usr/sbin/munin08:45:141
1554999178,7cyclictest2380-21ntp_states08:00:240
1554999172,12cyclictest12012-21kthreadcore08:45:190
1554999171,7cyclictest3845-21fschecks_count08:05:140
1554999171,3cyclictest0-21swapper/007:55:170
1554999171,2cyclictest131rcu_preempt08:25:160
1554999171,2cyclictest131rcu_preempt08:25:160
15549991712,3cyclictest0-21swapper/007:35:250
1554999166,3cyclictest131rcu_preempt07:45:160
1554999162,11cyclictest28016-21fschecks_count08:30:150
1554999151,11cyclictest15576-21date07:10:010
1563399133,3cyclictest0-21swapper/307:10:133
1563399131,9cyclictest0-21swapper/307:10:023
1563399122,7cyclictest0-21swapper/308:45:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional