You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-24 - 18:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Fri Jul 24, 2026 12:45:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2549673999721,56cyclictest0-21swapper/009:15:270
2549673999085,3cyclictest0-21swapper/007:30:020
2549673998823,34cyclictest0-21swapper/009:21:110
2549673998724,28cyclictest12-21ksoftirqd/011:25:030
2549673998526,32cyclictest0-21swapper/010:41:590
2549673998420,29cyclictest0-21swapper/011:50:020
2549673998420,29cyclictest0-21swapper/011:50:020
2549673998277,3cyclictest0-21swapper/011:25:030
2549673998014,36cyclictest0-21swapper/012:16:180
2549673997929,46cyclictest2614079-21ssh11:12:140
2549673997730,42cyclictest2612598-21ssh11:09:190
2549673997730,42cyclictest2612598-21ssh11:09:190
2549673997525,46cyclictest2571804-21ssh09:43:410
2549673997525,46cyclictest0-21swapper/009:12:010
2549673997432,38cyclictest2575186-21ssh09:50:220
2549673997423,47cyclictest2594398-21ssh10:30:460
2549673997414,37cyclictest0-21swapper/008:20:010
2549673997414,37cyclictest0-21swapper/008:20:010
2549673997340,29cyclictest364-21snmpd11:34:280
2549673997324,25cyclictest1715541-1kworker/0:0H12:15:020
2549673997231,11cyclictest2586753-21munin-run10:15:010
2549673997231,11cyclictest2586753-21munin-run10:15:010
2549673997029,36cyclictest2555912-21munin-run09:10:000
2549673997028,38cyclictest0-21swapper/009:28:030
2549673997023,18cyclictest0-21swapper/009:38:410
2549682996925,6cyclictest2564294-21sh09:27:422
2549673996928,10cyclictest2605300-21ssh10:54:080
2549673996924,12cyclictest2587321-21ssh10:15:460
2549673996828,35cyclictest2608486-21ssh11:00:240
2549673996823,41cyclictest2635854-21ssh11:59:100
2549673996822,15cyclictest2555409-21systemd-run09:00:010
2549673996822,15cyclictest2555409-21systemd-run09:00:000
2549682996725,38cyclictest2593687-21sh10:29:372
254968299672,19cyclictest2603346-21munin-run10:50:002
2549682996713,16cyclictest275-21dbus-daemon09:20:002
2549673996723,39cyclictest2624657-21ssh11:35:110
2549673996722,41cyclictest0-21swapper/012:30:020
2549673996721,10cyclictest2608277-21wc11:00:020
2549673996650,11cyclictest2549676-21latency_hist07:10:020
2549673996650,11cyclictest2549676-21latency_hist07:10:010
2549673996528,32cyclictest1-21systemd10:10:000
2549673996523,39cyclictest0-21swapper/011:25:020
2549673996521,39cyclictest2652926-21modprobe12:35:000
2549673996521,38cyclictest2578559-21ssh09:57:430
2549673996519,17cyclictest275-21dbus-daemon08:05:000
2549673996518,19cyclictest2654405-21rm12:37:570
254968799643,54cyclictest2632835-21kworker/u8:212:00:023
2549673996429,4cyclictest2602920-21ssh10:49:020
2549682996325,34cyclictest2631643-21latency_hist11:50:022
2549682996325,34cyclictest2631643-21latency_hist11:50:022
2549682996323,4cyclictest2617399-21sh11:19:262
2549682996322,12cyclictest30-21ksoftirqd/209:23:272
2549673996318,13cyclictest2555905-21diskmemload09:49:250
2549682996235,22cyclictest2555905-21diskmemload09:15:022
2549682996225,5cyclictest2577247-21sh09:55:012
2549673996235,23cyclictest2633895-21sh11:55:000
2549673996214,21cyclictest0-21swapper/012:08:140
2549673996214,21cyclictest0-21swapper/012:08:140
2549682996123,6cyclictest2633217-21sh11:53:282
2549682996118,11cyclictest30-21ksoftirqd/211:30:572
2549673996152,6cyclictest12-21ksoftirqd/008:09:130
2549673996152,6cyclictest12-21ksoftirqd/008:09:130
2549673996144,14cyclictest2550896-21sshd07:34:180
2549673996123,33cyclictest2553157-21latency_hist08:15:010
254968299604,12cyclictest2554335-21cron08:40:002
2549682996029,27cyclictest2551940-21cron07:55:002
2549682996022,10cyclictest30-21ksoftirqd/212:01:482
2549682996020,5cyclictest2555905-21diskmemload11:57:162
254967899604,10cyclictest23-21ksoftirqd/108:40:001
2549673996046,10cyclictest2554299-21sshd08:35:270
2549673996045,11cyclictest2551417-21sshd07:41:260
2549673996044,12cyclictest2580859-21sshd10:02:280
2549673996044,12cyclictest2580859-21sshd10:02:270
2549673996041,14cyclictest364-21snmpd10:29:110
2549673996040,15cyclictest0-21swapper/008:33:170
2549673996040,15cyclictest0-21swapper/008:33:160
2549673996032,10cyclictest2615650-21rm11:15:240
254968299592,24cyclictest311-21rs:main2
2549682995920,15cyclictest275-21dbus-daemon10:00:012
2549673995947,9cyclictest2551141-21sshd07:37:280
2549673995944,11cyclictest2552246-21munin-run08:00:020
2549673995944,11cyclictest2552246-21munin-run08:00:010
2549673995943,12cyclictest2554592-21sshd08:43:280
2549673995943,12cyclictest2553754-21sshd08:27:270
2549673995933,24cyclictest150-21pr/ttyS212:02:370
2549673995921,35cyclictest0-21swapper/010:37:270
2549682995820,9cyclictest30-21ksoftirqd/212:34:402
2549682995817,12cyclictest30-21ksoftirqd/211:09:442
2549682995817,12cyclictest30-21ksoftirqd/211:09:442
2549682995815,10cyclictest275-21dbus-daemon12:25:012
2549678995837,15cyclictest1-21systemd09:35:011
2549673995845,10cyclictest2648176-21munin-run12:25:000
2549673995831,22cyclictest2550509-21cat07:25:020
2549673995830,9cyclictest0-21swapper/009:31:170
254968799577,23cyclictest2586821-21sh10:15:023
254968799577,23cyclictest2586821-21sh10:15:023
254968299573,21cyclictest2643381-21munin-plugin-st12:15:002
2549682995725,9cyclictest30-21ksoftirqd/210:04:272
2549682995725,9cyclictest30-21ksoftirqd/210:04:272
2549682995718,12cyclictest3467602-21systemd-journal10:40:012
254967899579,5cyclictest131rcu_preempt11:35:021
2549673995720,4cyclictest2590522-21ssh10:22:460
254968299568,10cyclictest0-21swapper/212:20:022
254968299563,14cyclictest2552196-21modprobe08:00:002
254968299563,14cyclictest2552196-21modprobe08:00:002
2549678995613,34cyclictest23-21ksoftirqd/107:45:021
2549673995621,32cyclictest0-21swapper/011:42:270
254968299554,18cyclictest3467602-21systemd-journal11:45:002
254968299554,15cyclictest2605744-21cut10:55:012
2549682995523,9cyclictest30-21ksoftirqd/209:46:492
2549682995520,5cyclictest2590668-21sh10:23:052
2549682995517,8cyclictest2560093-1kworker/u9:209:42:112
2549678995539,12cyclictest1-21systemd09:45:011
2549673995535,16cyclictest2555632-21sshd09:04:270
254968299545,16cyclictest2650559-21modprobe12:30:002
2549682995446,5cyclictest2554626-21kworker/2:109:10:002
2549682995422,6cyclictest30-21ksoftirqd/209:34:162
2549682995421,4cyclictest2610193-21sh11:04:172
2549682995420,7cyclictest2625884-1kworker/u9:011:39:112
2549682995419,8cyclictest2555320-21sed08:55:022
2549682995419,6cyclictest30-21ksoftirqd/212:36:522
2549682995410,7cyclictest0-21swapper/210:57:232
2549678995418,12cyclictest23-21ksoftirqd/110:21:101
254968299534,10cyclictest2586729-21modprobe10:15:002
254968299534,10cyclictest2586729-21modprobe10:15:002
2549682995326,23cyclictest2595789-21sh10:34:012
2549682995324,25cyclictest2601114-21latency_hist10:45:032
2549678995311,6cyclictest131rcu_preempt12:40:001
2549673995320,29cyclictest0-21swapper/008:55:020
2549687995236,12cyclictest1-21systemd12:30:013
2549687995219,17cyclictest37-21ksoftirqd/312:09:433
2549687995219,17cyclictest37-21ksoftirqd/312:09:433
2549682995224,4cyclictest2549610-21latency_hist07:10:022
2549682995224,4cyclictest2549610-21latency_hist07:10:022
2549678995233,15cyclictest275-21dbus-daemon11:20:011
2549678995213,11cyclictest23-21ksoftirqd/111:47:141
2549678995213,11cyclictest23-21ksoftirqd/111:47:141
2549678995211,6cyclictest131rcu_preempt11:25:031
2549682995143,5cyclictest2554626-21kworker/2:109:36:052
2549682995143,5cyclictest2550704-21kworker/2:008:35:022
2549682995143,5cyclictest2550704-21kworker/2:008:35:022
2549682995125,5cyclictest30-21ksoftirqd/211:26:522
2549682995121,9cyclictest30-21ksoftirqd/212:09:462
2549682995121,9cyclictest30-21ksoftirqd/212:09:462
2549678995141,7cyclictest23-21ksoftirqd/112:06:161
2549678995141,7cyclictest23-21ksoftirqd/112:06:161
2549678995120,22cyclictest221rcuc/110:26:271
2549678995118,12cyclictest23-21ksoftirqd/110:20:011
2549673995143,5cyclictest2550107-21sshd07:18:260
2549673995139,9cyclictest2551680-21sshd07:49:210
2549682995041,5cyclictest2577254-21kworker/2:010:20:022
2549682995036,10cyclictest3467602-21systemd-journal07:13:272
2549682995015,5cyclictest0-21swapper/211:25:022
2549682995015,5cyclictest0-21swapper/211:25:022
254967899506,11cyclictest23-21ksoftirqd/112:22:011
2549678995021,9cyclictest23-21ksoftirqd/111:30:001
2549678995017,8cyclictest23-21ksoftirqd/112:14:081
2549678995013,14cyclictest23-21ksoftirqd/110:45:011
2549673995037,9cyclictest48-21kcompactd007:50:170
2549673995028,18cyclictest2553680-21date08:25:010
254968799497,26cyclictest2550894-21kworker/u8:107:35:033
2549687994939,7cyclictest37-21ksoftirqd/312:15:063
254968799493,34cyclictest2553258-21systemctl08:20:003
254968799493,34cyclictest2553258-21systemctl08:20:003
2549682994932,12cyclictest3467602-21systemd-journal07:44:592
254967899498,6cyclictest131rcu_preempt09:25:021
254967899491,8cyclictest23-21ksoftirqd/111:25:021
2549678994914,8cyclictest131rcu_preempt09:50:421
2549678994913,6cyclictest131rcu_preempt10:00:001
2549678994912,5cyclictest131rcu_preempt10:48:221
2549678994911,6cyclictest131rcu_preempt08:20:011
2549678994911,6cyclictest131rcu_preempt08:20:001
2549673994942,4cyclictest2554834-21sshd08:45:270
2549673994940,6cyclictest2549853-21sshd07:14:280
2549682994840,5cyclictest2554626-21kworker/2:109:00:022
2549682994840,5cyclictest2554626-21kworker/2:109:00:022
2549678994838,7cyclictest23-21ksoftirqd/111:35:331
2549678994837,9cyclictest23-21ksoftirqd/110:07:291
2549678994837,8cyclictest23-21ksoftirqd/111:11:311
254967899483,6cyclictest131rcu_preempt08:25:021
2549678994836,9cyclictest2594715-21sshd10:31:271
2549678994836,9cyclictest221rcuc/112:20:021
254967899481,17cyclictest0-21swapper/112:03:251
2549678994810,6cyclictest131rcu_preempt11:43:391
2549687994731,12cyclictest1-21systemd10:00:013
2549687994722,21cyclictest37-21ksoftirqd/309:16:163
2549682994733,8cyclictest2551170-21sh07:40:012
254967899475,6cyclictest131rcu_preempt11:50:451
254967899472,10cyclictest0-21swapper/110:52:431
254967899471,6cyclictest0-21swapper/109:47:481
254967899471,12cyclictest2558168-21cut09:15:001
2549678994710,6cyclictest131rcu_preempt09:30:011
2549687994627,10cyclictest37-21ksoftirqd/309:45:023
2549687994624,19cyclictest37-21ksoftirqd/310:50:013
254968799461,7cyclictest0-21swapper/312:34:543
2549678994611,8cyclictest23-21ksoftirqd/111:06:321
2549678994611,8cyclictest23-21ksoftirqd/111:06:321
2549687994535,7cyclictest37-21ksoftirqd/312:20:503
2549687994530,12cyclictest37-21ksoftirqd/310:27:393
2549687994530,11cyclictest275-21dbus-daemon11:50:013
2549687994530,11cyclictest275-21dbus-daemon11:50:013
254968799451,8cyclictest0-21swapper/310:37:003
254967899458,6cyclictest131rcu_preempt10:10:201
254967899458,6cyclictest131rcu_preempt10:10:191
254967899458,11cyclictest23-21ksoftirqd/109:17:381
2549678994531,10cyclictest23-21ksoftirqd/110:40:001
254967899452,4cyclictest131rcu_preempt11:02:271
2549678994515,6cyclictest131rcu_preempt10:59:271
2549687994432,10cyclictest37-21ksoftirqd/311:20:033
2549687994429,12cyclictest37-21ksoftirqd/309:45:203
254968799441,8cyclictest0-21swapper/312:38:413
2549687994417,21cyclictest131rcu_preempt11:10:023
2549687994417,21cyclictest131rcu_preempt11:10:023
254968799441,6cyclictest0-21swapper/311:34:343
2549682994436,5cyclictest2605777-21kworker/2:311:12:012
254968299442,7cyclictest3467602-21systemd-journal08:20:002
254968299442,7cyclictest3467602-21systemd-journal08:20:002
2549678994432,9cyclictest2555345-21sshd08:57:271
2549678994432,9cyclictest2555345-21sshd08:57:261
2549678994414,8cyclictest23-21ksoftirqd/111:58:501
2549678994413,8cyclictest23-21ksoftirqd/112:30:241
2549678994410,11cyclictest23-21ksoftirqd/112:26:151
2549687994331,8cyclictest37-21ksoftirqd/311:40:003
2549687994329,11cyclictest3467602-21systemd-journal12:05:003
2549687994328,11cyclictest275-21dbus-daemon09:35:023
2549687994321,19cyclictest37-21ksoftirqd/310:08:503
2549687994320,20cyclictest37-21ksoftirqd/311:12:513
2549687994319,20cyclictest361rcuc/308:30:013
254968799431,7cyclictest0-21swapper/309:50:233
254968799431,6cyclictest0-21swapper/309:37:443
2549682994323,15cyclictest275-21dbus-daemon07:50:012
2549678994333,6cyclictest23-21ksoftirqd/107:35:011
2549678994328,11cyclictest3467602-21systemd-journal08:26:271
2549678994313,5cyclictest131rcu_preempt09:37:111
2549678994310,6cyclictest131rcu_preempt07:55:011
2549678994310,5cyclictest131rcu_preempt10:01:261
2549678994310,5cyclictest131rcu_preempt10:01:251
254968799428,6cyclictest131rcu_preempt07:19:593
2549687994220,19cyclictest37-21ksoftirqd/307:15:023
254968799421,8cyclictest0-21swapper/310:03:453
254968799421,8cyclictest0-21swapper/310:03:453
2549687994216,6cyclictest131rcu_preempt09:22:183
254968799421,5cyclictest0-21swapper/310:54:233
2549682994234,5cyclictest2551714-21kworker/2:108:30:012
2549682994224,14cyclictest30-21ksoftirqd/210:10:012
2549682994219,17cyclictest275-21dbus-daemon09:04:592
2549678994225,13cyclictest275-21dbus-daemon08:00:011
2549678994225,13cyclictest275-21dbus-daemon08:00:001
2549687994131,6cyclictest2552208-21sendmail08:00:003
2549687994131,6cyclictest2552208-21sendmail08:00:003
2549687994129,9cyclictest37-21ksoftirqd/312:11:053
2549687994129,10cyclictest37-21ksoftirqd/308:35:013
2549687994129,10cyclictest37-21ksoftirqd/308:35:013
2549687994128,10cyclictest1-21systemd10:35:023
2549687994127,10cyclictest37-21ksoftirqd/311:26:113
2549687994124,13cyclictest371ksoftirqd/308:25:003
2549687994124,12cyclictest3467602-21systemd-journal10:40:023
254968799411,8cyclictest0-21swapper/311:40:093
254968799411,5cyclictest131rcu_preempt09:12:323
254968799411,5cyclictest0-21swapper/310:17:093
2549682994130,8cyclictest2550406-21cron07:25:012
2549682994123,13cyclictest1-21systemd08:45:002
254967899419,5cyclictest131rcu_preempt07:37:281
2549678994125,12cyclictest1-21systemd07:15:011
2549678994110,11cyclictest23-21ksoftirqd/107:30:011
254968799404,5cyclictest131rcu_preempt10:20:063
254968799403,5cyclictest131rcu_preempt08:05:273
254968799403,5cyclictest131rcu_preempt08:05:263
2549687994031,5cyclictest37-21ksoftirqd/311:50:223
2549687994018,19cyclictest37-21ksoftirqd/311:23:163
2549687994018,19cyclictest37-21ksoftirqd/311:23:163
254968799401,6cyclictest131rcu_preempt09:30:003
254968799401,6cyclictest131rcu_preempt07:45:023
254968799401,5cyclictest0-21swapper/311:04:123
2549682994029,8cyclictest2550113-21sshd07:19:272
2549682994027,10cyclictest30-21ksoftirqd/207:35:012
2549682994024,11cyclictest1-21systemd08:15:012
2549678994031,6cyclictest221rcuc/108:05:001
2549678994029,8cyclictest23-21ksoftirqd/108:33:161
2549678994029,8cyclictest23-21ksoftirqd/108:33:161
2549678994015,6cyclictest221rcuc/107:25:001
2549687993929,7cyclictest37-21ksoftirqd/307:30:023
254968799391,8cyclictest0-21swapper/310:56:413
2549682993924,11cyclictest3467602-21systemd-journal08:25:002
254967899395,6cyclictest131rcu_preempt07:10:021
254967899395,6cyclictest131rcu_preempt07:10:011
2549678993927,6cyclictest309-21in:imuxsock08:45:011
254967899392,13cyclictest23-21ksoftirqd/109:05:011
254967899386,9cyclictest23-21ksoftirqd/108:10:011
254967899386,9cyclictest23-21ksoftirqd/108:10:011
254967899385,9cyclictest23-21ksoftirqd/108:50:151
254967899384,5cyclictest131rcu_preempt07:45:271
2549678993826,9cyclictest1-21systemd07:20:011
2549687993727,7cyclictest37-21ksoftirqd/308:13:273
2549678993724,10cyclictest23-21ksoftirqd/109:10:011
2549678993721,11cyclictest23-21ksoftirqd/108:12:051
254968799364,6cyclictest131rcu_preempt07:25:013
2549687993622,9cyclictest275-21dbus-daemon08:50:003
2549682993622,10cyclictest30-21ksoftirqd/208:05:002
254967899364,9cyclictest23-21ksoftirqd/108:50:011
2549687993526,6cyclictest37-21ksoftirqd/309:00:023
2549687993526,6cyclictest37-21ksoftirqd/309:00:023
2549687993526,5cyclictest37-21ksoftirqd/308:45:023
254968799346,6cyclictest131rcu_preempt09:10:023
2549687993422,9cyclictest2552498-21pluginstate08:05:013
2549687993421,10cyclictest37-21ksoftirqd/307:52:273
254968799341,5cyclictest131rcu_preempt09:04:213
254968799334,5cyclictest131rcu_preempt08:50:333
254968799333,5cyclictest131rcu_preempt07:35:003
2549687993323,6cyclictest2-21kthreadd07:49:593
2549687993220,9cyclictest2554316-21sshd08:39:273
2549682993220,9cyclictest30-21ksoftirqd/208:10:012
2549682993220,9cyclictest30-21ksoftirqd/208:10:012
254968299311,5cyclictest131rcu_preempt08:50:032
2549687992914,12cyclictest2549708-21sed07:10:023
2549687992914,12cyclictest2549708-21sed07:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional