You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-04 - 23:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sat Jul 04, 2026 12:45:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
185081991423,7cyclictest131rcu_preempt10:15:023
185081991423,7cyclictest131rcu_preempt10:15:023
185071999285,3cyclictest150-21pr/ttyS208:00:030
18507199760,36cyclictest0-21swapper/008:05:010
18507199760,36cyclictest0-21swapper/008:05:010
18507199754,54cyclictest236351-21ssh10:45:020
185071997429,5cyclictest0-21swapper/010:38:360
18507199736,41cyclictest0-21swapper/011:35:260
185074997247,13cyclictest189232-21kworker/u8:108:30:021
185071997152,16cyclictest16950irq/224-800000010:52:500
185074997018,14cyclictest275-21dbus-daemon09:40:011
185081996826,38cyclictest275-21dbus-daemon12:30:013
185074996825,5cyclictest214449-21sh09:59:151
185074996713,12cyclictest275-21dbus-daemon12:00:021
18507199668,33cyclictest194127-21rm09:15:420
185081996523,4cyclictest202701-21sh09:34:193
185081996523,4cyclictest202701-21sh09:34:193
18507199650,35cyclictest0-21swapper/010:35:020
18508199649,16cyclictest290084-21sendmail12:40:013
185074996426,33cyclictest274-21cron12:30:011
185074996423,6cyclictest235088-21sh10:42:341
18507499641,22cyclictest238473-21ssh10:50:011
185081996323,10cyclictest0-21swapper/310:25:003
185074996319,5cyclictest0-21swapper/111:24:011
18507199632,44cyclictest250799-21ssh11:15:570
185081996226,32cyclictest190060-21pluginstate08:45:013
185081996224,4cyclictest234346-21sh10:40:453
185081996219,9cyclictest275-21dbus-daemon10:40:013
185081996219,4cyclictest191335-21diskmemload09:20:253
185081996219,4cyclictest191335-21diskmemload09:20:253
185074996230,29cyclictest120-21kswapd010:25:011
185074996223,7cyclictest188995-21sshd08:24:441
185074996223,4cyclictest272378-21sh12:01:281
185074996218,5cyclictest259645-21latency_hist11:35:021
185071996210,15cyclictest282095-21ssh12:22:510
185081996120,8cyclictest37-21ksoftirqd/308:50:003
185074996120,5cyclictest191335-21diskmemload11:46:181
18507199619,18cyclictest218811-21ssh10:08:110
185071996115,8cyclictest12-21ksoftirqd/010:13:470
18508199604,20cyclictest212483-21unin-run09:55:013
185081996025,10cyclictest37-21ksoftirqd/310:47:173
185081996018,10cyclictest37-21ksoftirqd/307:45:443
185081996017,8cyclictest238881-1kworker/u9:010:52:503
185081996010,14cyclictest48-21kcompactd011:55:213
185074996025,32cyclictest221965-21sh10:15:011
185074996025,29cyclictest3467602-21systemd-journal09:09:591
185074996021,4cyclictest255565-21sh11:25:551
185074996020,5cyclictest191335-21diskmemload09:17:311
185074996019,5cyclictest261275-21rm11:38:191
18507199608,48cyclictest0-21swapper/011:41:410
185071996030,26cyclictest212465-21munin-run09:55:000
18508199593,14cyclictest195978-21cron09:19:593
185081995924,9cyclictest37-21ksoftirqd/309:39:583
185081995920,10cyclictest37-21ksoftirqd/311:29:443
185071995945,10cyclictest3467602-21systemd-journal11:30:010
185071995938,16cyclictest16850irq/134-800000010:26:290
185071995938,16cyclictest16850irq/134-800000010:26:280
185071995916,37cyclictest217224-21munin-run10:05:000
185071995915,8cyclictest286280-21ssh12:31:440
185081995822,4cyclictest189262-21latency_hist08:30:023
18508199582,17cyclictest229068-21cut10:30:003
18508199582,17cyclictest229068-21cut10:30:003
185081995819,8cyclictest37-21ksoftirqd/310:07:013
185077995850,5cyclictest176277-21kworker/2:110:37:432
185074995824,4cyclictest262815-21sh11:41:271
185074995823,7cyclictest23-21ksoftirqd/109:43:421
185074995823,7cyclictest23-21ksoftirqd/109:43:421
18507199589,16cyclictest199741-21rm09:27:490
18507199588,15cyclictest248300-21ssh11:10:310
185071995815,7cyclictest12-21ksoftirqd/009:24:010
185071995815,7cyclictest12-21ksoftirqd/009:24:000
185071995812,41cyclictest272777-21ssh12:02:290
185071995812,41cyclictest245523-21ssh11:05:000
185081995728,25cyclictest275414-21sh12:08:133
185081995726,28cyclictest37-21ksoftirqd/310:56:303
185081995726,28cyclictest37-21ksoftirqd/310:56:293
185074995721,4cyclictest227151-21sshd10:25:431
185074995721,4cyclictest227151-21sshd10:25:431
18507199575,35cyclictest276519-21ssh12:10:130
185071995712,13cyclictest213025-21ssh09:55:440
185081995624,4cyclictest246959-21sh11:07:593
185081995624,4cyclictest246959-21sh11:07:593
185081995622,5cyclictest309-21in:imuxsock07:30:003
185081995622,5cyclictest309-21in:imuxsock07:30:003
185081995622,4cyclictest191335-21diskmemload12:21:243
185081995622,12cyclictest37-21ksoftirqd/308:40:033
18508199562,20cyclictest261971-21cron11:40:003
185081995619,5cyclictest208983-21sh09:47:223
185074995639,13cyclictest275-21dbus-daemon12:39:321
185074995621,6cyclictest23-21ksoftirqd/111:54:591
185074995617,9cyclictest23-21ksoftirqd/110:05:491
185071995627,13cyclictest266429-21ssh11:49:160
185071995626,3cyclictest0-21swapper/012:18:360
185071995617,9cyclictest191335-21diskmemload12:36:440
185071995615,36cyclictest223588-21ssh10:18:300
185071995615,36cyclictest223588-21ssh10:18:290
185081995525,7cyclictest245416-21sh11:04:483
185081995522,10cyclictest37-21ksoftirqd/312:13:153
185081995520,6cyclictest190932-21latency_hist09:00:013
185081995519,4cyclictest249447-21sh11:13:153
185081995518,10cyclictest273913-21cut12:05:013
185074995522,4cyclictest199186-21sh09:26:271
185074995522,13cyclictest23-21ksoftirqd/109:47:491
185074995520,4cyclictest231690-21sh10:35:181
185071995513,7cyclictest12-21ksoftirqd/011:31:130
185071995511,14cyclictest224617-21ssh10:20:160
185077995432,17cyclictest275-21dbus-daemon10:50:012
18507199548,41cyclictest285108-21ssh12:29:240
185071995432,17cyclictest185540-21sshd07:18:430
18507199543,14cyclictest202072-21ssh09:32:500
18507199543,14cyclictest202072-21ssh09:32:490
185071995421,28cyclictest253365-21ssh11:21:180
185071995417,7cyclictest12-21ksoftirqd/011:09:400
185071995417,7cyclictest12-21ksoftirqd/011:09:390
18508199532,18cyclictest254928-21munin-run11:25:003
185077995344,6cyclictest205618-21kworker/2:210:08:052
185077995334,14cyclictest275-21dbus-daemon07:50:012
185077995322,19cyclictest275-21dbus-daemon08:00:002
18507499534,13cyclictest1-21systemd09:55:011
185074995324,4cyclictest215609-21sh10:01:161
185074995320,4cyclictest202975-21latency_hist09:35:021
185074995320,4cyclictest202975-21latency_hist09:35:011
185074995319,6cyclictest23-21ksoftirqd/111:02:421
18507199537,16cyclictest270107-21ssh11:56:520
18507199537,15cyclictest192561-21ssh09:12:290
18507199537,15cyclictest192561-21ssh09:12:290
185071995315,14cyclictest189020-21systemd-run08:25:010
185081995222,7cyclictest37-21ksoftirqd/308:00:013
185081995220,7cyclictest37-21ksoftirqd/310:14:133
185081995217,9cyclictest37-21ksoftirqd/311:31:533
185081995216,9cyclictest37-21ksoftirqd/308:05:023
185081995216,9cyclictest37-21ksoftirqd/308:05:013
185077995244,5cyclictest205618-21kworker/2:210:27:232
185077995244,5cyclictest205618-21kworker/2:210:27:222
185077995244,5cyclictest205618-21kworker/2:210:18:482
185077995244,5cyclictest205618-21kworker/2:210:18:482
185077995244,5cyclictest205618-21kworker/2:210:04:412
185077995229,16cyclictest3467602-21systemd-journal09:39:592
185077995214,9cyclictest131rcu_preempt09:18:072
18507799521,47cyclictest250198-21cron11:14:592
18507499527,39cyclictest221rcuc/108:15:021
18507499527,39cyclictest221rcuc/108:15:021
185074995223,6cyclictest23-21ksoftirqd/110:18:561
185074995223,6cyclictest23-21ksoftirqd/110:18:561
185074995221,7cyclictest23-21ksoftirqd/112:20:261
185071995222,15cyclictest235977-21ssh10:44:440
18507199521,36cyclictest12-21ksoftirqd/008:47:440
185071995210,37cyclictest205960-21ssh09:40:460
185071995210,37cyclictest205960-21ssh09:40:450
185081995122,10cyclictest37-21ksoftirqd/309:10:283
185081995122,10cyclictest37-21ksoftirqd/309:10:283
18508199512,20cyclictest264646-21ssh11:45:043
185077995143,5cyclictest238490-21kworker/2:011:16:102
185077995143,5cyclictest205618-21kworker/2:209:55:042
185077995143,5cyclictest188480-21kworker/2:009:14:042
185077995143,5cyclictest188480-21kworker/2:009:14:032
185077995143,5cyclictest176277-21kworker/2:112:26:322
185077995142,5cyclictest205618-21kworker/2:209:46:522
185077995142,5cyclictest176277-21kworker/2:111:21:172
185077995135,12cyclictest1-21systemd08:45:012
185077995134,12cyclictest275-21dbus-daemon12:20:012
185077995126,19cyclictest3467602-21systemd-journal11:50:022
185077995121,21cyclictest275-21dbus-daemon11:05:012
185074995122,4cyclictest278399-21latency_hist12:15:021
185074995115,8cyclictest190530-21sshd08:53:441
185071995136,11cyclictest1-21systemd07:26:430
185071995136,11cyclictest1-21systemd07:26:430
185071995124,11cyclictest241405-21ssh10:55:520
185071995124,11cyclictest241405-21ssh10:55:510
185081995021,5cyclictest185564-21latency_hist07:20:033
185081995018,8cyclictest37-21ksoftirqd/311:53:443
185077995042,6cyclictest176277-21kworker/2:111:32:172
185077995042,5cyclictest205618-21kworker/2:210:31:062
185077995041,6cyclictest176277-21kworker/2:111:50:082
185077995041,5cyclictest205618-21kworker/2:209:51:082
185074995022,7cyclictest23-21ksoftirqd/112:19:151
185074995019,8cyclictest23-21ksoftirqd/109:00:011
185074995018,6cyclictest23-21ksoftirqd/112:30:341
185074995017,9cyclictest23-21ksoftirqd/110:34:381
18507199504,18cyclictest191335-21diskmemload09:35:440
18507199501,17cyclictest191478-21cut09:10:020
18508199494,11cyclictest214824-21munin-run10:00:003
185077994941,5cyclictest205618-21kworker/2:210:22:152
185077994941,5cyclictest176277-21kworker/2:111:58:502
185077994940,6cyclictest188480-21kworker/2:009:21:002
185077994940,6cyclictest188480-21kworker/2:009:20:592
185077994940,5cyclictest176277-21kworker/2:112:08:462
185077994939,5cyclictest176277-21kworker/2:111:39:522
185074994921,10cyclictest23-21ksoftirqd/111:00:011
185074994921,10cyclictest23-21ksoftirqd/111:00:001
185074994920,5cyclictest274918-21sh12:07:001
185074994920,4cyclictest188686-21kworker/u8:208:20:001
185074994916,11cyclictest23-21ksoftirqd/107:50:001
185074994915,9cyclictest23-21ksoftirqd/107:35:011
185074994913,9cyclictest23-21ksoftirqd/107:30:021
185074994913,9cyclictest23-21ksoftirqd/107:30:011
185071994934,11cyclictest190810-21munin-plugin-st09:00:000
185071994930,16cyclictest16850irq/134-800000011:52:030
185081994821,5cyclictest37-21ksoftirqd/310:02:243
185081994819,5cyclictest207836-21sed09:45:023
185081994819,5cyclictest207836-21sed09:45:023
185077994840,5cyclictest238490-21kworker/2:010:58:212
185077994840,5cyclictest238490-21kworker/2:010:58:212
185077994839,5cyclictest188480-21kworker/2:009:28:332
185077994838,5cyclictest176277-21kworker/2:112:39:442
185077994837,6cyclictest176277-21kworker/2:107:26:432
185077994837,6cyclictest176277-21kworker/2:107:26:432
18507499485,8cyclictest23-21ksoftirqd/111:08:431
18507499485,8cyclictest23-21ksoftirqd/111:08:431
185074994811,11cyclictest3467602-21systemd-journal08:35:001
185071994831,13cyclictest189750-21sshd08:39:430
18508199471,6cyclictest0-21swapper/311:19:123
185081994714,21cyclictest37-21ksoftirqd/308:15:023
185081994713,11cyclictest37-21ksoftirqd/312:31:463
185077994739,5cyclictest176277-21kworker/2:107:20:442
185077994738,5cyclictest176277-21kworker/2:112:12:182
185077994738,5cyclictest176277-21kworker/2:111:44:422
185074994720,5cyclictest191335-21diskmemload11:10:231
185071994733,11cyclictest186902-21sshd07:42:430
185071994731,13cyclictest190530-21sshd08:53:430
185071994731,12cyclictest187396-21sshd07:50:080
185071994718,13cyclictest274698-21ssh12:06:330
185071994714,29cyclictest16850irq/134-800000009:45:580
18508199463,4cyclictest131rcu_preempt09:29:423
185081994618,8cyclictest37-21ksoftirqd/310:30:563
18507799466,30cyclictest3467602-21systemd-journal08:20:012
185077994638,5cyclictest176277-21kworker/2:112:21:102
185077994638,5cyclictest176277-21kworker/2:108:45:332
185077994638,5cyclictest176277-21kworker/2:107:50:452
185077994637,6cyclictest205618-21kworker/2:210:10:152
185077994637,6cyclictest176277-21kworker/2:110:51:032
185077994629,13cyclictest189738-21sshd08:35:432
185077994623,17cyclictest275-21dbus-daemon09:00:002
185071994633,10cyclictest190031-21sshd08:44:430
18508199454,29cyclictest361rcuc/308:15:033
18508199454,29cyclictest361rcuc/308:15:023
18508199452,4cyclictest131rcu_preempt07:40:003
18508199452,3cyclictest131rcu_preempt09:05:013
18508199452,3cyclictest131rcu_preempt09:05:003
185077994537,5cyclictest188480-21kworker/2:009:31:182
185077994537,5cyclictest188480-21kworker/2:009:31:172
185077994537,5cyclictest176277-21kworker/2:110:40:512
185077994536,5cyclictest176277-21kworker/2:108:25:192
185077994529,10cyclictest185548-21cron07:20:002
185071994510,31cyclictest0-21swapper/007:34:080
18508199444,3cyclictest131rcu_preempt12:20:023
18508199444,3cyclictest131rcu_preempt07:25:003
18508199441,5cyclictest0-21swapper/311:43:053
18507799446,25cyclictest186641-21modprobe07:40:002
185077994434,6cyclictest176277-21kworker/2:109:41:032
185077994434,6cyclictest176277-21kworker/2:109:41:032
185074994418,7cyclictest23-21ksoftirqd/109:20:331
185074994418,7cyclictest23-21ksoftirqd/109:20:321
18507499441,4cyclictest131rcu_preempt11:16:171
18507499441,4cyclictest131rcu_preempt10:52:551
185081994322,15cyclictest1-21systemd07:45:003
18508199431,13cyclictest188379-21cat08:10:023
185077994335,5cyclictest176277-21kworker/2:112:30:312
18507499439,5cyclictest131rcu_preempt09:10:191
18507499439,5cyclictest131rcu_preempt09:10:181
185074994330,10cyclictest3467602-21systemd-journal07:15:001
185071994335,4cyclictest0-21swapper/007:50:010
18507199431,10cyclictest186657-21munin-run07:40:000
185081994227,11cyclictest189133-21sed08:25:033
185077994228,9cyclictest0-21swapper/208:35:002
18507499427,3cyclictest131rcu_preempt07:45:001
185071994229,10cyclictest185854-21/usr/sbin/munin07:25:020
185071994210,13cyclictest12-21ksoftirqd/008:35:030
185077994129,7cyclictest176277-21kworker/2:107:15:012
18507799411,5cyclictest0-21swapper/211:09:352
18507799411,5cyclictest0-21swapper/211:09:352
185071994121,16cyclictest364-21snmpd07:10:030
185071994112,25cyclictest0-21swapper/008:18:350
18508199404,5cyclictest131rcu_preempt09:10:013
18508199401,4cyclictest131rcu_preempt08:55:023
185077994032,5cyclictest176277-21kworker/2:111:26:482
185077994031,6cyclictest187949-21cron08:05:002
185077994031,6cyclictest187949-21cron08:05:002
185077994027,9cyclictest186925-21modprobe07:45:002
185074994027,10cyclictest1-21systemd07:25:001
18507499401,3cyclictest131rcu_preempt07:55:001
185071994029,8cyclictest3467602-21systemd-journal08:06:400
185071994027,10cyclictest0-21swapper/008:15:010
185071994027,10cyclictest0-21swapper/008:15:010
18508199391,3cyclictest131rcu_preempt08:32:433
185077993931,5cyclictest176277-21kworker/2:109:10:022
185077993931,5cyclictest176277-21kworker/2:108:15:002
185077993931,5cyclictest176277-21kworker/2:108:15:002
18507499399,6cyclictest131rcu_preempt08:08:431
185071993928,8cyclictest191046-21sshd09:00:340
185071993928,8cyclictest191046-21sshd09:00:340
18507499385,4cyclictest131rcu_preempt08:03:441
18507499385,4cyclictest131rcu_preempt08:03:431
185074993823,10cyclictest275-21dbus-daemon07:40:001
18508199378,21cyclictest187146-21kworker/u8:007:55:023
185077993717,13cyclictest275-21dbus-daemon08:55:012
18507499371,5cyclictest131rcu_preempt07:10:021
18507499371,4cyclictest131rcu_preempt08:50:001
185081993618,13cyclictest1-21systemd07:15:023
18507799364,24cyclictest189013-21munin-plugin-st08:25:002
185077993626,8cyclictest30-21ksoftirqd/212:01:332
185077993623,8cyclictest188235-21latency_hist08:10:012
185077993623,10cyclictest186480-21sed07:35:012
18507499365,5cyclictest131rcu_preempt09:05:011
18507499365,5cyclictest131rcu_preempt09:05:011
18507499361,8cyclictest23-21ksoftirqd/107:57:181
185074993527,5cyclictest185564-21latency_hist07:20:001
185074993524,7cyclictest189739-21sshd08:36:441
18507199358,5cyclictest131rcu_preempt08:30:010
18508199343,3cyclictest131rcu_preempt07:10:013
18507799343,21cyclictest191104-21latency_hist09:05:012
18507799343,21cyclictest191104-21latency_hist09:05:012
18507199347,19cyclictest185070-21sort07:10:020
185081993321,9cyclictest186462-21sort07:35:013
185077993117,10cyclictest0-21swapper/207:10:012
18507499281,4cyclictest131rcu_preempt08:44:591
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional