You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-01 - 14:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Wed Apr 01, 2026 12:45:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3186599668,9cyclictest131rcu_preempt11:20:141
31865996010,12cyclictest131rcu_preempt10:05:131
3186599575,29cyclictest131rcu_preempt09:25:131
3187599545,11cyclictest27-21ksoftirqd/209:55:132
3185599522,38cyclictest9779-21ssh12:05:130
3185599508,37cyclictest656-21ssh10:30:120
3188999473,24cyclictest28366irq/58-eth007:25:133
3186599474,30cyclictest17558-21diskmemload10:20:161
3188999454,27cyclictest28366irq/58-eth010:45:133
31875994533,4cyclictest15118-21ssh11:50:142
31865994325,9cyclictest131rcu_preempt08:55:131
3188999423,27cyclictest28366irq/58-eth009:55:143
3186599422,30cyclictest22343-21ssh11:00:121
3186599421,34cyclictest11297-21ssh10:00:131
3186599421,32cyclictest22298-21ssh12:30:131
31855994211,24cyclictest12-21ksoftirqd/009:05:130
3186599412,32cyclictest2242-21rm09:55:151
3185599411,34cyclictest31410-21ssh11:05:150
3188999401,21cyclictest28366irq/58-eth011:05:143
3186599403,6cyclictest21-21ksoftirqd/108:00:111
3186599402,8cyclictest131rcu_preempt08:45:141
3188999391,23cyclictest28366irq/58-eth012:10:133
3186599390,6cyclictest131rcu_preempt08:35:131
3188999372,19cyclictest28366irq/58-eth009:35:143
3187599371,27cyclictest4304-21ssh12:20:132
3186599372,25cyclictest18700-21ssh10:40:141
31855993620,10cyclictest31838-21cyclictest07:20:130
3187599352,29cyclictest29583-21sshd11:40:122
3187599351,27cyclictest14719-21ssh09:43:272
3186599352,22cyclictest17558-21diskmemload10:25:141
3188999342,17cyclictest28366irq/58-eth011:30:143
3187599342,23cyclictest12645-21ssh09:25:132
3186599340,26cyclictest17955-21ssh09:10:131
3185599343,21cyclictest6178-21ssh11:45:140
3185599341,25cyclictest9624-21ssh10:35:130
3188999331,18cyclictest28366irq/58-eth010:15:123
3188999330,19cyclictest28366irq/58-eth012:35:123
3187599323,23cyclictest0-21swapper/210:09:502
3185599324,18cyclictest30717-21ssh09:35:140
3185599322,22cyclictest26988-21ssh09:15:140
3188999313,17cyclictest28366irq/58-eth011:20:143
3186599312,21cyclictest15404-21ssh11:13:361
3186599311,25cyclictest25521-21ssh09:50:141
3185599314,22cyclictest11625-21ssh11:30:150
3186599302,24cyclictest3627-21diskmemload09:20:131
31865993014,7cyclictest21-21ksoftirqd/108:25:141
31865993011,10cyclictest131rcu_preempt07:35:151
3185599304,8cyclictest19849-21cut12:10:210
3188999292,16cyclictest28366irq/58-eth009:25:133
3188999291,18cyclictest28366irq/58-eth009:20:133
3187599293,21cyclictest0-21swapper/210:54:082
3187599292,11cyclictest4924-21ssh09:37:372
3186599296,7cyclictest131rcu_preempt10:39:101
3185599295,7cyclictest9100-21apt-key07:20:010
3185599294,7cyclictest12-21ksoftirqd/010:15:300
3185599293,8cyclictest25522-21ssh09:50:140
3185599292,20cyclictest13314-21ssh10:55:150
31855992917,6cyclictest31838-21cyclictest08:05:140
3185599291,17cyclictest25066-21grep10:25:220
31889992817,4cyclictest0-21swapper/307:45:123
3187599282,20cyclictest0-21swapper/211:20:152
3187599282,20cyclictest0-21swapper/209:20:142
3187599281,16cyclictest29257-21ssh09:50:282
3186599283,18cyclictest4246-21ssh10:50:131
3186599281,23cyclictest20649-21ssh11:35:131
3186599281,22cyclictest16495-21ssh09:45:161
3186599281,17cyclictest839-21munin-node07:45:131
31855992823,3cyclictest24156-21apt-get11:55:140
3188999273,15cyclictest28366irq/58-eth009:05:133
3188999272,18cyclictest28366irq/58-eth007:55:133
3188999271,11cyclictest28366irq/58-eth010:00:133
3187599273,18cyclictest0-21swapper/211:13:372
3187599272,21cyclictest27-21ksoftirqd/212:25:142
3187599272,19cyclictest0-21swapper/209:30:142
3187599272,19cyclictest0-21swapper/207:20:042
3186599272,18cyclictest8967-21apt-get07:20:031
3186599271,21cyclictest21-21ksoftirqd/110:15:121
3186599271,19cyclictest31221-21ssh12:35:141
3185599275,5cyclictest12-21ksoftirqd/010:00:250
3185599275,5cyclictest12-21ksoftirqd/008:20:180
3185599274,3cyclictest12-21ksoftirqd/009:10:210
31855992721,3cyclictest23668-21apt-get07:35:140
31855992717,8cyclictest823-21apt-get07:45:120
3188999262,18cyclictest0-21swapper/312:28:463
3187599262,9cyclictest31190-21apt-get12:35:122
3187599262,12cyclictest27698-21sh12:15:142
3187599261,13cyclictest0-21swapper/211:05:132
31875992610,8cyclictest6056-21ssh10:15:122
3187599260,13cyclictest9815-21cut10:35:172
31865992612,3cyclictest0-21swapper/108:40:121
3185599263,5cyclictest12-21ksoftirqd/010:45:010
3185599262,5cyclictest12-21ksoftirqd/009:40:130
31855992621,3cyclictest12628-21apt-get09:25:130
3185599261,10cyclictest4091-21ssh11:42:580
3185599260,22cyclictest0-21swapper/008:00:120
3187599252,15cyclictest22346irq/40-dwc2_hso07:10:142
3187599252,14cyclictest924-21snmpd11:25:152
31875992510,4cyclictest27-21ksoftirqd/210:42:482
3186599254,14cyclictest0-21swapper/107:40:111
3186599252,17cyclictest21-21ksoftirqd/107:25:141
3186599251,5cyclictest0-21swapper/111:50:151
3185599255,7cyclictest24782-21grep11:35:300
3185599255,6cyclictest12-21ksoftirqd/009:45:260
3185599255,5cyclictest12-21ksoftirqd/011:15:200
3185599254,8cyclictest32374-21cut08:15:220
3185599254,5cyclictest12-21ksoftirqd/012:30:150
3185599254,5cyclictest12-21ksoftirqd/012:15:130
3185599254,5cyclictest12-21ksoftirqd/010:10:250
3185599254,4cyclictest12-21ksoftirqd/011:00:260
3185599254,4cyclictest12-21ksoftirqd/009:30:220
3185599253,6cyclictest4015-21ssh12:36:000
3185599253,5cyclictest4415-21kthreadcore09:20:190
3185599253,5cyclictest12-21ksoftirqd/007:40:150
3185599253,11cyclictest4432-21ntp_states12:00:260
3185599252,5cyclictest12-21ksoftirqd/010:25:020
31855992518,4cyclictest20396-21apt-get10:05:140
3185599250,10cyclictest3898-21latency_hist10:50:020
3188999244,8cyclictest28466irq/59-eth011:00:133
3187599249,9cyclictest131rcu_preempt11:01:282
3187599244,12cyclictest22346irq/40-dwc2_hso08:15:122
31875992414,7cyclictest4484-21sshd10:13:352
31875992413,8cyclictest0-21swapper/211:45:142
31875992411,8cyclictest27-21ksoftirqd/207:30:192
3187599241,17cyclictest22346irq/40-dwc2_hso09:00:142
31875992410,2cyclictest131rcu_preempt08:30:242
3186599243,16cyclictest746-21ssh12:00:131
3186599241,20cyclictest18705-21ssh12:10:121
3185599244,6cyclictest12-21ksoftirqd/012:25:210
3185599244,5cyclictest12-21ksoftirqd/008:35:150
3185599244,5cyclictest12-21ksoftirqd/007:50:300
3185599244,4cyclictest12-21ksoftirqd/011:10:270
3185599243,6cyclictest4474-21fschecks_count12:20:170
3185599242,12cyclictest4684-21irqcore10:50:190
3185599241,4cyclictest12-21ksoftirqd/008:25:130
31855992411,4cyclictest131rcu_preempt11:20:160
3185599240,8cyclictest3643-21kthreadcore11:25:200
3188999233,13cyclictest0-21swapper/311:15:133
31889992313,7cyclictest0-21swapper/309:45:153
31889992311,8cyclictest0-21swapper/311:45:003
3188999231,14cyclictest0-21swapper/312:30:133
3187599238,8cyclictest131rcu_preempt08:45:172
3187599233,8cyclictest131rcu_preempt08:50:032
3187599233,16cyclictest0-21swapper/209:47:372
3187599232,16cyclictest0-21swapper/211:57:142
3187599232,16cyclictest0-21swapper/210:03:502
3187599232,15cyclictest0-21swapper/210:55:072
3187599232,14cyclictest22346irq/40-dwc2_hso08:40:122
31875992313,4cyclictest0-21swapper/208:20:262
31875992311,5cyclictest8969-21ssh12:04:322
3187599231,13cyclictest924-21snmpd07:50:142
31875992310,10cyclictest27-21ksoftirqd/212:05:182
31875992310,10cyclictest21028-21cron08:40:012
3186599233,11cyclictest21-21ksoftirqd/109:30:131
3186599232,3cyclictest131rcu_preempt11:55:151
31865992311,9cyclictest24587-21cron08:10:011
3186599231,16cyclictest9798-21ssh12:05:141
3186599231,11cyclictest761-21find08:15:281
31865992310,9cyclictest27681-21ssh10:45:141
3185599237,10cyclictest0-21swapper/007:10:130
3185599235,5cyclictest12-21ksoftirqd/009:55:260
3185599234,4cyclictest12-21ksoftirqd/007:30:290
3185599234,4cyclictest0-21swapper/011:50:190
3185599233,5cyclictest3761-21sed08:55:190
3185599233,5cyclictest12-21ksoftirqd/007:25:220
3185599230,5cyclictest12-21ksoftirqd/008:30:020
3188999228,8cyclictest28366irq/58-eth009:15:143
3188999223,9cyclictest0-21swapper/308:20:133
3188999223,13cyclictest0-21swapper/311:13:373
3188999223,12cyclictest0-21swapper/310:20:173
3188999221,12cyclictest28366irq/58-eth010:35:143
31889992210,9cyclictest0-21swapper/310:30:213
3187599229,6cyclictest171-21systemd-network10:23:492
3187599228,11cyclictest2293-21kthreadcore07:45:182
3187599226,7cyclictest131rcu_preempt12:13:252
3187599224,4cyclictest27-21ksoftirqd/210:30:262
3187599221,6cyclictest2559866ssh10:25:222
31875992213,7cyclictest0-21swapper/211:35:272
31875992212,7cyclictest27-21ksoftirqd/212:30:132
31875992211,9cyclictest1258-21ssh10:46:542
31875992211,8cyclictest27-21ksoftirqd/207:15:142
3187599221,10cyclictest19404-21kthreadcore11:15:202
31875992210,6cyclictest131rcu_preempt09:18:342
31875992210,4cyclictest27-21ksoftirqd/211:30:542
3186599222,3cyclictest131rcu_preempt11:40:231
3186599222,14cyclictest839-21munin-node09:40:151
3186599221,9cyclictest27661-21cut09:15:181
3186599221,4cyclictest0-21swapper/112:15:241
3186599221,16cyclictest0-21swapper/110:10:131
31865992210,4cyclictest0-21swapper/112:20:121
31865992210,3cyclictest0-21swapper/111:32:331
3185599224,5cyclictest12-21ksoftirqd/009:00:210
3185599224,5cyclictest12-21ksoftirqd/007:55:210
3185599224,4cyclictest12-21ksoftirqd/008:50:190
3185599222,6cyclictest12-21ksoftirqd/008:45:210
31855992216,3cyclictest21267-21apt-get08:40:110
3188999213,12cyclictest0-21swapper/309:43:263
3188999212,7cyclictest0-21swapper/311:55:133
31889992110,7cyclictest28366irq/58-eth010:50:273
31889992110,4cyclictest0-21swapper/309:50:253
3187599219,3cyclictest11784-21kthreadcore07:55:192
3187599218,4cyclictest27-21ksoftirqd/208:25:192
3187599218,10cyclictest28207-21sshd07:38:072
3187599217,3cyclictest131rcu_preempt08:00:132
3187599212,13cyclictest131rcu_preempt08:55:182
31875992112,7cyclictest27-21ksoftirqd/208:05:152
3187599211,10cyclictest20618-21cut09:10:232
31875992110,4cyclictest27-21ksoftirqd/209:05:132
3186599218,3cyclictest0-21swapper/109:05:171
3186599212,14cyclictest21208-21ssh12:29:121
3186599211,3cyclictest131rcu_preempt11:15:121
31865992111,7cyclictest13295-21apt-get10:55:141
3185599213,5cyclictest12-21ksoftirqd/008:10:290
3188999209,8cyclictest0-21swapper/310:28:563
3188999203,11cyclictest0-21swapper/309:30:323
31889992011,7cyclictest0-21swapper/310:40:163
3187599205,4cyclictest27-21ksoftirqd/207:25:262
3187599204,13cyclictest24931-21grep08:10:152
3187599200,8cyclictest27-21ksoftirqd/207:40:182
3186599205,6cyclictest2481-21kthreadcore10:30:201
3186599201,3cyclictest131rcu_preempt08:10:261
31865992013,5cyclictest0-21swapper/111:25:141
3188999196,6cyclictest0-21swapper/311:45:143
3188999193,11cyclictest0-21swapper/310:13:363
3188999192,12cyclictest0-21swapper/311:35:133
3188999192,11cyclictest0-21swapper/308:45:143
3188999192,10cyclictest0-21swapper/307:40:123
3188999191,11cyclictest28366irq/58-eth010:05:133
3188999191,10cyclictest28366irq/58-eth008:50:113
31889991910,6cyclictest0-21swapper/312:03:283
3186599196,3cyclictest131rcu_preempt09:35:231
3186599191,9cyclictest32056-21irqrtprio11:05:201
3186599190,5cyclictest53146usb-storage11:45:181
3188999183,7cyclictest28366irq/58-eth011:50:143
3188999183,10cyclictest0-21swapper/310:55:083
3188999182,10cyclictest0-21swapper/308:30:153
3188999182,10cyclictest0-21swapper/307:20:053
3188999181,12cyclictest0-21swapper/308:30:013
31889991810,6cyclictest0-21swapper/312:16:243
31889991810,6cyclictest0-21swapper/311:25:133
3186599189,7cyclictest9039-21ntpq07:50:231
3186599185,6cyclictest131rcu_preempt07:15:261
3186599184,4cyclictest131rcu_preempt07:30:141
3186599181,4cyclictest0-21swapper/108:50:121
31865991810,6cyclictest11657-21apt-get08:30:131
3188999179,6cyclictest0-21swapper/312:08:573
3188999179,6cyclictest0-21swapper/309:12:463
3188999172,9cyclictest0-21swapper/307:35:153
3188999172,10cyclictest0-21swapper/312:20:153
3188999172,10cyclictest0-21swapper/308:10:133
3188999172,10cyclictest0-21swapper/307:50:143
3187599170,9cyclictest31838-21cyclictest07:10:022
3186599170,6cyclictest131rcu_preempt07:10:231
3188999162,11cyclictest0-21swapper/307:10:153
3188999161,10cyclictest0-21swapper/309:00:223
3186599162,9cyclictest0-21swapper/108:20:141
3186599161,4cyclictest131rcu_preempt07:55:171
3186599161,12cyclictest11321-21kthreadcore09:00:221
3188999151,7cyclictest0-21swapper/308:45:023
31889991510,3cyclictest0-21swapper/308:05:143
3188999149,3cyclictest33-21ksoftirqd/307:15:123
3188999141,8cyclictest28366irq/58-eth007:30:123
3188999141,8cyclictest0-21swapper/308:00:163
3188999140,10cyclictest0-21swapper/308:55:153
3188999132,9cyclictest0-21swapper/308:15:143
3186599138,3cyclictest31777-21latency_hist07:10:021
3185599131,9cyclictest31883-21latency_hist07:10:010
3188999122,7cyclictest0-21swapper/308:35:273
3188999101,6cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional