You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-27 - 16:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Wed May 27, 2026 12:45:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
645651997931,23cyclictest676854-21ssh10:05:150
645660997366,3cyclictest30-21ksoftirqd/208:25:022
645660997366,3cyclictest30-21ksoftirqd/208:25:012
645656997142,23cyclictest23-21ksoftirqd/108:55:021
645656997142,23cyclictest23-21ksoftirqd/108:55:011
645660996844,7cyclictest683328-21kworker/2:312:16:332
645660996836,6cyclictest648472-21kworker/2:010:31:442
645660996836,6cyclictest648472-21kworker/2:010:31:442
64566599677,12cyclictest37-21ksoftirqd/311:00:003
645651996730,18cyclictest0-21swapper/009:12:160
645665996617,6cyclictest131rcu_preempt10:55:023
645665996526,9cyclictest37-21ksoftirqd/312:40:003
64565199647,44cyclictest723571-21kworker/0:012:15:550
645651996410,37cyclictest0-21swapper/011:45:560
645651996410,37cyclictest0-21swapper/011:45:560
645660996333,8cyclictest648472-21kworker/2:012:28:202
645665996215,12cyclictest37-21ksoftirqd/307:45:013
64566099628,32cyclictest652024-21ssh09:10:022
64565699629,21cyclictest646879-21kworker/u8:308:15:031
645651996241,5cyclictest16850irq/134-800000012:15:030
645651996230,16cyclictest12-21ksoftirqd/007:20:030
645651996224,7cyclictest12-21ksoftirqd/010:30:020
645651996216,41cyclictest12-21ksoftirqd/009:40:440
645651996211,34cyclictest131rcu_preempt07:40:000
64566599617,12cyclictest37-21ksoftirqd/311:59:563
64566599617,11cyclictest37-21ksoftirqd/310:45:013
64566599617,11cyclictest37-21ksoftirqd/310:45:003
645665996112,5cyclictest131rcu_preempt09:40:023
64565199617,5cyclictest131rcu_preempt10:15:280
64565199615,35cyclictest131rcu_preempt10:36:210
64565199614,36cyclictest131rcu_preempt09:21:160
645651996124,11cyclictest12-21ksoftirqd/011:28:200
64565199611,36cyclictest12-21ksoftirqd/010:55:130
64566599603,10cyclictest37-21ksoftirqd/311:10:013
64566599603,10cyclictest37-21ksoftirqd/311:10:013
645665996013,6cyclictest131rcu_preempt12:20:003
645665996010,6cyclictest131rcu_preempt11:40:023
645665996010,6cyclictest131rcu_preempt11:40:023
64565199607,31cyclictest131rcu_preempt11:05:020
645651996013,8cyclictest12-21ksoftirqd/011:08:400
645651996013,8cyclictest12-21ksoftirqd/011:08:400
645651996010,6cyclictest131rcu_preempt09:10:020
64566599598,10cyclictest37-21ksoftirqd/311:25:023
64566599595,9cyclictest37-21ksoftirqd/309:00:013
645665995912,6cyclictest131rcu_preempt11:48:533
645665995912,6cyclictest131rcu_preempt11:48:533
645665995912,6cyclictest131rcu_preempt10:20:033
645660995941,7cyclictest648472-21kworker/2:010:12:382
64565199598,10cyclictest12-21ksoftirqd/009:35:190
645651995932,18cyclictest12-21ksoftirqd/009:17:460
645651995922,7cyclictest12-21ksoftirqd/011:14:160
645651995918,5cyclictest131rcu_preempt08:50:000
645665995816,8cyclictest37-21ksoftirqd/309:27:563
645665995814,6cyclictest131rcu_preempt09:35:023
645665995814,5cyclictest131rcu_preempt12:15:013
645665995814,5cyclictest131rcu_preempt09:56:013
645665995813,6cyclictest131rcu_preempt11:19:133
645665995813,5cyclictest131rcu_preempt12:30:503
645665995813,5cyclictest131rcu_preempt12:30:503
645651995825,17cyclictest669099-21ssh09:48:200
64566599577,9cyclictest37-21ksoftirqd/310:26:553
64566599577,6cyclictest131rcu_preempt10:00:053
64566599575,10cyclictest37-21ksoftirqd/311:14:033
645665995732,9cyclictest37-21ksoftirqd/310:35:373
64566599571,9cyclictest37-21ksoftirqd/311:45:013
645665995719,12cyclictest37-21ksoftirqd/311:30:013
645665995712,6cyclictest131rcu_preempt11:30:233
645665995711,5cyclictest131rcu_preempt10:34:313
645665995711,5cyclictest131rcu_preempt10:34:313
645665995711,5cyclictest131rcu_preempt10:13:003
645665995711,5cyclictest131rcu_preempt08:10:023
64565199571,4cyclictest131rcu_preempt09:55:370
64565199571,39cyclictest720757-21ssh11:43:380
64566599569,6cyclictest131rcu_preempt10:15:183
645665995614,6cyclictest131rcu_preempt10:09:513
645665995614,5cyclictest131rcu_preempt10:50:013
645665995612,8cyclictest131rcu_preempt07:10:023
645665995611,6cyclictest131rcu_preempt09:16:303
645660995637,8cyclictest648472-21kworker/2:012:05:392
64565199568,8cyclictest131rcu_preempt10:25:020
645651995619,32cyclictest16850irq/134-800000009:55:020
645651995618,34cyclictest12-21ksoftirqd/009:34:230
64566599559,5cyclictest131rcu_preempt11:55:013
64566599557,5cyclictest131rcu_preempt12:27:213
645665995513,6cyclictest131rcu_preempt09:41:153
645665995513,5cyclictest131rcu_preempt11:04:413
645665995512,6cyclictest131rcu_preempt07:20:013
645665995512,5cyclictest131rcu_preempt12:02:233
645665995511,5cyclictest131rcu_preempt08:00:023
645665995511,5cyclictest131rcu_preempt08:00:013
645660995546,6cyclictest648472-21kworker/2:009:34:302
645660995537,5cyclictest648472-21kworker/2:011:58:532
645660995524,22cyclictest30-21ksoftirqd/211:49:192
645660995524,22cyclictest30-21ksoftirqd/211:49:192
645656995536,14cyclictest3467602-21systemd-journal11:50:001
645656995536,14cyclictest3467602-21systemd-journal11:50:001
64565199557,4cyclictest131rcu_preempt11:30:250
645651995524,27cyclictest12-21ksoftirqd/010:05:000
645651995524,26cyclictest19350irq/34-mmc112:24:150
645651995524,26cyclictest19350irq/34-mmc112:24:150
645651995519,18cyclictest16850irq/134-800000011:20:570
645651995518,6cyclictest16850irq/134-800000012:38:190
645651995518,18cyclictest16850irq/134-800000010:40:110
645651995518,18cyclictest16850irq/134-800000010:40:110
64565199551,5cyclictest131rcu_preempt08:40:010
64565199551,5cyclictest131rcu_preempt08:40:000
645665995414,5cyclictest131rcu_preempt09:50:533
645665995413,5cyclictest131rcu_preempt09:21:163
645665995411,9cyclictest131rcu_preempt09:45:223
645665995411,6cyclictest131rcu_preempt12:20:193
645665995411,6cyclictest131rcu_preempt12:20:193
64566099549,6cyclictest131rcu_preempt11:00:012
645660995446,5cyclictest648472-21kworker/2:012:04:592
645660995437,13cyclictest1-21systemd09:15:012
645660995434,7cyclictest648472-21kworker/2:012:10:042
64565199547,4cyclictest131rcu_preempt10:10:580
645651995445,5cyclictest12-21ksoftirqd/010:35:010
645651995445,5cyclictest12-21ksoftirqd/010:35:000
64565199542,6cyclictest131rcu_preempt11:20:010
645660995345,5cyclictest648472-21kworker/2:011:35:002
645660995345,5cyclictest648472-21kworker/2:009:26:202
645660995344,5cyclictest683328-21kworker/2:311:21:212
645660995344,5cyclictest648472-21kworker/2:009:54:462
645660995343,6cyclictest648472-21kworker/2:009:17:082
645660995337,6cyclictest683328-21kworker/2:311:17:472
645660995336,7cyclictest683328-21kworker/2:310:49:252
645660995324,20cyclictest30-21ksoftirqd/207:35:012
645651995318,18cyclictest16850irq/134-800000012:34:360
645651995318,18cyclictest16850irq/134-800000012:34:350
645651995318,18cyclictest16850irq/134-800000009:27:530
645651995313,6cyclictest131rcu_preempt12:00:000
645665995217,8cyclictest131rcu_preempt09:12:283
645665995213,6cyclictest37-21ksoftirqd/308:20:013
645665995213,6cyclictest131rcu_preempt12:10:023
645665995212,6cyclictest131rcu_preempt08:51:233
645665995212,6cyclictest131rcu_preempt08:51:223
645660995244,5cyclictest683328-21kworker/2:311:26:022
645660995244,5cyclictest683328-21kworker/2:311:14:122
645660995244,5cyclictest648472-21kworker/2:011:41:192
645660995244,5cyclictest648472-21kworker/2:010:07:462
645660995244,5cyclictest648472-21kworker/2:009:47:592
645660995244,5cyclictest648472-21kworker/2:009:20:352
645660995243,6cyclictest648472-21kworker/2:010:02:552
64565199527,5cyclictest131rcu_preempt11:39:080
64565199527,5cyclictest131rcu_preempt11:39:080
645651995236,12cyclictest19350irq/34-mmc108:25:030
645651995217,18cyclictest16850irq/134-800000012:06:340
645651995215,6cyclictest16850irq/134-800000010:51:100
645651995215,18cyclictest16850irq/134-800000011:51:110
64565199521,41cyclictest131rcu_preempt10:48:580
64566599519,9cyclictest37-21ksoftirqd/307:15:013
645665995115,7cyclictest37-21ksoftirqd/308:45:023
645660995144,4cyclictest648472-21kworker/2:010:35:282
645660995143,5cyclictest683328-21kworker/2:311:02:562
645660995143,5cyclictest648472-21kworker/2:010:17:212
645660995143,5cyclictest648472-21kworker/2:009:56:002
645660995143,5cyclictest648472-21kworker/2:008:50:022
645660995143,5cyclictest647940-21kworker/2:308:19:232
64565199513,5cyclictest131rcu_preempt08:20:000
645651995116,7cyclictest12-21ksoftirqd/008:05:000
645651995114,5cyclictest131rcu_preempt08:00:020
645651995114,5cyclictest131rcu_preempt08:00:020
64566599506,9cyclictest37-21ksoftirqd/308:35:023
645665995016,7cyclictest131rcu_preempt07:25:013
645665995014,5cyclictest131rcu_preempt08:04:093
645665995012,5cyclictest131rcu_preempt09:05:023
645665995011,6cyclictest131rcu_preempt08:40:013
645665995011,6cyclictest131rcu_preempt08:40:003
645660995044,4cyclictest648472-21kworker/2:011:35:062
645660995044,4cyclictest648472-21kworker/2:011:35:052
645660995042,5cyclictest683328-21kworker/2:311:07:032
645660995042,5cyclictest683328-21kworker/2:311:07:032
645660995042,5cyclictest683328-21kworker/2:310:50:232
645660995042,5cyclictest648472-21kworker/2:011:52:052
645660995042,5cyclictest648472-21kworker/2:009:42:272
645660995042,5cyclictest648472-21kworker/2:008:59:072
645660995041,6cyclictest648472-21kworker/2:010:22:342
645660995041,6cyclictest648472-21kworker/2:009:38:192
645660995041,5cyclictest683328-21kworker/2:310:41:192
645660995041,5cyclictest683328-21kworker/2:310:41:192
645656995033,13cyclictest275-21dbus-daemon07:20:011
645656995028,7cyclictest131rcu_preempt11:25:181
645651995014,8cyclictest12-21ksoftirqd/008:55:030
645651995014,8cyclictest12-21ksoftirqd/008:55:030
64566599499,5cyclictest131rcu_preempt08:30:013
645660994941,5cyclictest648472-21kworker/2:012:37:282
645660994941,5cyclictest648472-21kworker/2:012:20:042
645660994941,5cyclictest648472-21kworker/2:012:20:042
645660994941,5cyclictest648472-21kworker/2:010:25:552
645660994941,5cyclictest633566-21kworker/2:107:50:062
645660994941,5cyclictest633566-21kworker/2:107:24:592
645660994940,5cyclictest648472-21kworker/2:012:34:012
645660994940,5cyclictest648472-21kworker/2:012:34:012
64565699493,3cyclictest131rcu_preempt10:21:301
645656994913,11cyclictest131rcu_preempt09:40:011
64565199491,5cyclictest131rcu_preempt09:05:020
64566599484,6cyclictest131rcu_preempt07:30:013
645665994813,6cyclictest131rcu_preempt08:50:013
64566599481,15cyclictest37-21ksoftirqd/307:35:013
645665994811,5cyclictest131rcu_preempt08:10:033
645660994841,4cyclictest647940-21kworker/2:308:02:502
645660994839,6cyclictest648472-21kworker/2:008:54:562
645660994839,6cyclictest648472-21kworker/2:008:54:552
645660994817,23cyclictest30-21ksoftirqd/207:50:002
645656994835,9cyclictest3467602-21systemd-journal09:35:021
64565699483,3cyclictest131rcu_preempt11:04:021
64565199486,6cyclictest131rcu_preempt08:59:590
64565199483,13cyclictest275-21dbus-daemon12:04:590
645651994817,9cyclictest12-21ksoftirqd/007:15:010
64566599479,6cyclictest37-21ksoftirqd/309:10:033
64566599478,6cyclictest131rcu_preempt07:40:003
64566599478,11cyclictest37-21ksoftirqd/307:55:003
645660994739,5cyclictest633566-21kworker/2:107:57:522
645660994739,5cyclictest633566-21kworker/2:107:57:522
645660994739,5cyclictest633566-21kworker/2:107:12:112
64565699473,3cyclictest131rcu_preempt11:35:171
64565699473,3cyclictest131rcu_preempt11:35:171
64565699472,3cyclictest131rcu_preempt11:17:141
645651994728,14cyclictest3467602-21systemd-journal12:25:030
645651994715,10cyclictest12-21ksoftirqd/007:30:000
645660994639,4cyclictest647940-21kworker/2:308:11:482
645660994638,5cyclictest647940-21kworker/2:308:07:462
645660994638,5cyclictest633566-21kworker/2:107:44:052
645660994638,5cyclictest633566-21kworker/2:107:29:342
64565699461,3cyclictest131rcu_preempt12:22:171
64565699461,3cyclictest131rcu_preempt12:22:171
64565199468,6cyclictest131rcu_preempt08:10:010
645660994537,5cyclictest648472-21kworker/2:008:41:272
645660994537,5cyclictest648472-21kworker/2:008:36:382
645660994537,5cyclictest648472-21kworker/2:008:36:372
645656994535,6cyclictest712348-21munin-plugin-st11:25:001
645656994513,9cyclictest131rcu_preempt09:15:021
645651994511,6cyclictest131rcu_preempt08:45:020
645660994436,5cyclictest648472-21kworker/2:008:27:342
645660994436,5cyclictest633566-21kworker/2:107:38:462
645656994432,10cyclictest23-21ksoftirqd/110:02:111
645656994430,10cyclictest275-21dbus-daemon07:15:011
645656994429,11cyclictest275-21dbus-daemon10:40:011
645656994428,12cyclictest23-21ksoftirqd/109:20:241
645656994428,11cyclictest23-21ksoftirqd/112:09:591
64565699442,3cyclictest131rcu_preempt10:12:111
64565699442,3cyclictest131rcu_preempt09:44:311
64565699440,31cyclictest0-21swapper/112:30:021
64565199446,6cyclictest131rcu_preempt07:45:000
645651994436,4cyclictest16950irq/224-800000007:50:010
64565199443,20cyclictest645654-21latency_hist07:10:010
64566599438,5cyclictest131rcu_preempt07:50:013
645665994329,9cyclictest309-21in:imuxsock08:20:033
645665994329,9cyclictest309-21in:imuxsock08:20:033
645660994335,5cyclictest648472-21kworker/2:009:04:002
645660994335,5cyclictest633566-21kworker/2:107:18:192
64565699433,14cyclictest651838-21ssh09:10:011
645656994329,10cyclictest120-21kswapd012:15:021
645656994310,9cyclictest131rcu_preempt11:00:021
645651994330,9cyclictest1-21systemd07:35:000
645656994226,11cyclictest23-21ksoftirqd/108:05:001
64565199429,5cyclictest131rcu_preempt08:15:020
64565199426,33cyclictest12-21ksoftirqd/008:30:010
645651994235,4cyclictest16950irq/224-800000008:25:010
645660994133,5cyclictest648472-21kworker/2:008:30:192
64565699419,5cyclictest131rcu_preempt09:55:021
64565699414,5cyclictest131rcu_preempt10:42:581
64565699414,5cyclictest131rcu_preempt10:42:571
645656994130,7cyclictest23-21ksoftirqd/110:35:011
645656994130,7cyclictest23-21ksoftirqd/110:35:001
645656994129,9cyclictest275-21dbus-daemon10:10:001
645656994126,11cyclictest3467602-21systemd-journal12:04:231
64565699412,3cyclictest131rcu_preempt09:56:511
645656994122,14cyclictest275-21dbus-daemon11:35:011
645656994118,19cyclictest23-21ksoftirqd/107:30:011
64565699411,4cyclictest131rcu_preempt12:15:121
645651994112,7cyclictest131rcu_preempt07:55:000
64565699403,3cyclictest131rcu_preempt11:10:241
645656994028,10cyclictest23-21ksoftirqd/111:44:131
645656994027,10cyclictest23-21ksoftirqd/108:50:001
645656994019,8cyclictest131rcu_preempt08:35:011
64565199404,8cyclictest12-21ksoftirqd/008:35:000
645651994012,18cyclictest131rcu_preempt07:20:220
645656993931,5cyclictest669786-21latency_hist09:50:021
64565699393,10cyclictest0-21swapper/110:55:021
645656993927,9cyclictest275-21dbus-daemon10:20:011
64565699392,3cyclictest131rcu_preempt10:47:021
645656993916,8cyclictest23-21ksoftirqd/109:25:371
64565699391,3cyclictest131rcu_preempt08:20:021
64565699386,6cyclictest346771850lldpd11:06:051
64565699386,6cyclictest346771850lldpd11:06:051
64565699383,3cyclictest131rcu_preempt07:55:011
645656993827,8cyclictest23-21ksoftirqd/112:40:011
645656993825,10cyclictest23-21ksoftirqd/111:56:261
645656993824,11cyclictest23-21ksoftirqd/109:16:461
64565699382,3cyclictest131rcu_preempt12:30:481
64565699382,3cyclictest131rcu_preempt12:30:481
64565699379,8cyclictest131rcu_preempt08:40:011
64565699379,8cyclictest131rcu_preempt08:40:001
64565699376,5cyclictest131rcu_preempt08:25:031
64565699376,5cyclictest131rcu_preempt08:25:021
645656993724,10cyclictest725304-21ssh11:53:501
64565699371,3cyclictest131rcu_preempt07:40:011
64566099362,7cyclictest0-21swapper/207:10:002
645656993628,5cyclictest685833-21sh10:25:131
645656993625,6cyclictest23-21ksoftirqd/109:00:021
645656993623,8cyclictest23-21ksoftirqd/107:50:001
64565699362,29cyclictest364-21snmpd07:34:591
64565699353,5cyclictest651716-21latency_hist09:05:011
645656993423,7cyclictest0-21swapper/107:10:011
645656993421,9cyclictest23-21ksoftirqd/107:56:291
645656993421,9cyclictest23-21ksoftirqd/107:56:291
645656993415,15cyclictest131rcu_preempt08:45:011
645656993315,7cyclictest131rcu_preempt08:10:001
64565699313,3cyclictest131rcu_preempt07:45:021
645656993114,6cyclictest131rcu_preempt07:25:011
64565699301,3cyclictest131rcu_preempt08:26:231
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional