You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-02 - 22:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Sun Aug 02, 2026 12:45:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1297115991096,89cyclictest0-21swapper/009:55:020
1297115999588,5cyclictest12-21ksoftirqd/007:50:030
1297115998474,8cyclictest0-21swapper/010:10:030
1297115997874,2cyclictest0-21swapper/010:35:030
129711599781,74cyclictest0-21swapper/011:05:010
1297130997627,10cyclictest1298601-21cron07:40:013
1297122997564,6cyclictest0-21swapper/207:10:012
129712299707,8cyclictest30-21ksoftirqd/209:45:012
129711599665,13cyclictest12-21ksoftirqd/009:30:010
129712299655,11cyclictest30-21ksoftirqd/212:20:002
129711599656,40cyclictest131rcu_preempt10:54:080
1297122996415,7cyclictest30-21ksoftirqd/210:30:012
1297115996452,7cyclictest12-21ksoftirqd/010:40:000
129713099636,51cyclictest1387688-21ssh12:10:033
129711599636,8cyclictest12-21ksoftirqd/012:35:010
129712299625,11cyclictest30-21ksoftirqd/212:13:072
1297115996246,12cyclictest1340565-21sed10:30:010
1297122996120,11cyclictest30-21ksoftirqd/212:05:022
1297122996120,11cyclictest30-21ksoftirqd/212:05:022
1297122996114,12cyclictest30-21ksoftirqd/211:45:022
129711599612,5cyclictest131rcu_preempt10:41:060
129711599612,5cyclictest131rcu_preempt10:41:050
129713099602,25cyclictest1347450-21kworker/u8:111:20:023
129712299595,8cyclictest30-21ksoftirqd/212:30:012
1297115995922,8cyclictest12-21ksoftirqd/011:49:410
1297122995810,11cyclictest30-21ksoftirqd/211:59:592
129711599588,34cyclictest284-21systemd-logind09:15:010
129711599583,40cyclictest1371348-21ssh11:35:030
129712299573,16cyclictest30-21ksoftirqd/211:35:022
129712299572,17cyclictest30-21ksoftirqd/209:30:302
129711599573,5cyclictest1373526-21modprobe11:40:000
129711599572,4cyclictest131rcu_preempt12:06:210
129711599571,9cyclictest1307918-21systemctl09:20:010
1297115995711,35cyclictest12-21ksoftirqd/012:00:520
1297115995711,35cyclictest12-21ksoftirqd/012:00:510
1297115995710,5cyclictest131rcu_preempt10:20:350
129713099565,4cyclictest131rcu_preempt11:55:013
129712299568,9cyclictest30-21ksoftirqd/210:00:002
129712299567,10cyclictest30-21ksoftirqd/209:47:542
129712299564,6cyclictest30-21ksoftirqd/209:00:012
1297122995616,8cyclictest30-21ksoftirqd/212:37:232
1297122995616,8cyclictest30-21ksoftirqd/212:37:232
1297122995614,13cyclictest30-21ksoftirqd/208:20:012
1297122995611,15cyclictest30-21ksoftirqd/211:05:002
1297118995641,10cyclictest1-21systemd10:45:011
1297118995641,10cyclictest1-21systemd10:45:001
129711599564,5cyclictest131rcu_preempt11:00:010
129711599562,5cyclictest12-21ksoftirqd/012:22:540
1297115995622,30cyclictest151rcuc/009:50:020
129711599561,17cyclictest12-21ksoftirqd/009:10:010
129712299552,11cyclictest30-21ksoftirqd/208:50:012
129712299551,16cyclictest1364525-21sh11:20:332
129711899558,7cyclictest131rcu_preempt11:40:001
129711599553,34cyclictest131rcu_preempt12:27:550
1297115995527,13cyclictest1309523-21ssh09:23:180
1297115995521,7cyclictest12-21ksoftirqd/011:56:200
129712299546,9cyclictest30-21ksoftirqd/210:38:192
129712299544,10cyclictest30-21ksoftirqd/212:09:332
129712299543,11cyclictest30-21ksoftirqd/209:21:212
1297122995415,9cyclictest30-21ksoftirqd/210:50:012
1297118995438,12cyclictest1-21systemd10:55:021
1297118995413,10cyclictest131rcu_preempt09:34:301
129711599549,12cyclictest12-21ksoftirqd/009:40:030
1297115995425,14cyclictest1313504-21ssh09:31:410
1297115995418,30cyclictest16850irq/134-800000010:45:180
1297115995410,6cyclictest131rcu_preempt12:40:020
1297115995410,6cyclictest131rcu_preempt12:40:010
129713099531,4cyclictest131rcu_preempt09:05:023
129712299539,8cyclictest30-21ksoftirqd/209:52:022
129712299532,15cyclictest30-21ksoftirqd/210:11:332
129712299532,13cyclictest30-21ksoftirqd/212:30:432
129712299531,15cyclictest30-21ksoftirqd/209:26:242
1297122995310,12cyclictest30-21ksoftirqd/211:36:502
1297118995338,9cyclictest1401015-21sshd12:38:341
1297118995338,9cyclictest1401015-21sshd12:38:331
129711599536,5cyclictest131rcu_preempt11:06:340
1297115995325,13cyclictest1315475-21ssh09:35:550
129711599532,13cyclictest3467602-21systemd-journal09:00:000
129711599531,4cyclictest131rcu_preempt09:57:090
129712299528,12cyclictest1342344-21ssh10:33:442
129712299526,11cyclictest30-21ksoftirqd/210:20:342
129712299525,8cyclictest30-21ksoftirqd/209:10:022
129712299525,11cyclictest30-21ksoftirqd/211:52:412
1297122995232,14cyclictest275-21dbus-daemon11:30:012
129712299521,7cyclictest1360178-21ssh11:11:272
129712299521,7cyclictest1360178-21ssh11:11:272
1297122995216,8cyclictest131rcu_preempt09:17:112
1297122995211,11cyclictest30-21ksoftirqd/210:55:032
129712299521,10cyclictest1316992-21ssh09:39:272
1297115995242,8cyclictest151rcuc/011:20:010
1297115995212,5cyclictest131rcu_preempt12:15:220
1297130995131,15cyclictest275-21dbus-daemon11:25:013
1297122995142,6cyclictest1329201-21kworker/2:211:06:362
129712299514,14cyclictest30-21ksoftirqd/210:55:252
129712299514,11cyclictest30-21ksoftirqd/211:16:152
129712299513,8cyclictest30-21ksoftirqd/210:41:342
129712299513,8cyclictest30-21ksoftirqd/210:41:332
129712299511,15cyclictest30-21ksoftirqd/212:25:002
1297118995133,13cyclictest275-21dbus-daemon07:20:011
1297115995143,5cyclictest19350irq/34-mmc107:20:030
1297115995113,6cyclictest131rcu_preempt10:05:000
129712299507,7cyclictest30-21ksoftirqd/210:06:312
1297122995010,10cyclictest30-21ksoftirqd/210:00:472
129711599509,6cyclictest131rcu_preempt10:15:000
129711599504,13cyclictest12-21ksoftirqd/008:00:010
129711599502,11cyclictest12-21ksoftirqd/010:20:000
1297115995013,6cyclictest131rcu_preempt11:25:020
1297115995010,5cyclictest131rcu_preempt11:29:590
1297130994939,6cyclictest1378215-21munin-plugin-st11:49:593
129712299498,6cyclictest131rcu_preempt07:55:002
129712299498,11cyclictest30-21ksoftirqd/209:15:012
129712299495,10cyclictest30-21ksoftirqd/208:35:002
129712299494,8cyclictest30-21ksoftirqd/210:17:332
1297122994941,5cyclictest1297885-21kworker/2:207:43:422
129712299494,10cyclictest30-21ksoftirqd/211:46:162
1297122994937,8cyclictest1-21systemd08:30:012
1297122994937,8cyclictest1-21systemd08:30:002
1297122994910,7cyclictest30-21ksoftirqd/208:15:022
1297118994936,10cyclictest23-21ksoftirqd/108:15:001
129711599499,7cyclictest12-21ksoftirqd/012:11:420
129711599497,10cyclictest12-21ksoftirqd/011:55:030
1297115994916,30cyclictest12-21ksoftirqd/011:15:010
1297115994916,30cyclictest12-21ksoftirqd/011:15:000
1297130994814,7cyclictest131rcu_preempt12:20:013
129712299488,10cyclictest30-21ksoftirqd/207:40:012
129712299486,9cyclictest30-21ksoftirqd/208:25:012
129712299485,8cyclictest30-21ksoftirqd/207:30:002
1297122994838,6cyclictest1289450-21kworker/2:007:12:502
129712299483,11cyclictest30-21ksoftirqd/208:55:022
129712299483,11cyclictest30-21ksoftirqd/208:55:012
129712299481,4cyclictest0-21swapper/208:45:022
129711899483,4cyclictest131rcu_preempt11:40:031
129711599486,23cyclictest3468525-21sendmail-mta08:05:020
129711599485,8cyclictest12-21ksoftirqd/011:40:130
129711599482,7cyclictest12-21ksoftirqd/008:55:030
129711599482,7cyclictest12-21ksoftirqd/008:55:020
1297130994729,14cyclictest275-21dbus-daemon10:10:013
129712299477,7cyclictest30-21ksoftirqd/207:20:012
1297122994739,5cyclictest1282734-21kworker/2:307:33:342
129711599472,7cyclictest12-21ksoftirqd/008:20:020
129712299466,8cyclictest30-21ksoftirqd/208:05:012
1297122994639,4cyclictest1300189-21kworker/2:109:01:442
1297122994639,4cyclictest1297885-21kworker/2:207:45:542
1297122994639,4cyclictest1282734-21kworker/2:308:07:092
1297118994631,11cyclictest3467602-21systemd-journal12:24:401
1297118994630,12cyclictest275-21dbus-daemon07:35:011
129711599469,5cyclictest131rcu_preempt08:10:010
129711599467,5cyclictest131rcu_preempt08:15:020
129711599466,5cyclictest131rcu_preempt08:50:000
1297115994629,10cyclictest275-21dbus-daemon08:40:010
129713099454,6cyclictest131rcu_preempt10:50:003
1297122994538,5cyclictest1300189-21kworker/2:108:37:262
1297122994537,5cyclictest1282734-21kworker/2:307:56:582
1297122994537,5cyclictest1282734-21kworker/2:307:21:462
1297118994527,13cyclictest23-21ksoftirqd/112:05:021
1297118994527,13cyclictest23-21ksoftirqd/112:05:021
129711899452,3cyclictest131rcu_preempt11:30:011
1297118994523,3cyclictest23-21ksoftirqd/111:20:011
129711599458,6cyclictest131rcu_preempt08:25:020
129711599456,7cyclictest12-21ksoftirqd/007:55:010
1297130994433,6cyclictest1399443-21latency_hist12:35:033
1297130994432,7cyclictest309-21in:imuxsock11:05:013
1297130994429,11cyclictest0-21swapper/309:15:023
129713099442,8cyclictest1317321-21munin-run09:40:013
1297130994418,17cyclictest37-21ksoftirqd/310:45:023
1297118994434,7cyclictest23-21ksoftirqd/110:40:001
129711899443,3cyclictest131rcu_preempt11:50:021
1297118994431,10cyclictest3467602-21systemd-journal11:35:001
1297118994426,14cyclictest0-21swapper/109:39:221
129711599447,7cyclictest12-21ksoftirqd/007:10:020
1297130994334,7cyclictest37-21ksoftirqd/310:51:583
1297130994331,8cyclictest3467602-21systemd-journal10:38:353
1297130994330,10cyclictest1333530-21systemd-run10:15:013
1297130994328,11cyclictest1401730-21wc12:40:013
1297130994328,11cyclictest1401730-21wc12:40:013
1297130994328,11cyclictest1327455-21sshd10:01:333
1297130994328,11cyclictest1321942-21sh09:50:003
1297130994326,14cyclictest1306353-21sshd09:16:343
129711899433,3cyclictest131rcu_preempt10:31:181
1297118994326,14cyclictest23-21ksoftirqd/109:45:021
1297118994322,16cyclictest275-21dbus-daemon11:05:021
129711899431,3cyclictest131rcu_preempt12:00:021
129711599434,7cyclictest12-21ksoftirqd/008:45:010
129711599433,6cyclictest1298623-21munin-run07:40:000
1297115994335,5cyclictest12-21ksoftirqd/009:05:010
1297130994229,10cyclictest3467602-21systemd-journal10:59:293
1297130994228,10cyclictest3467602-21systemd-journal10:31:343
1297130994224,13cyclictest3467602-21systemd-journal08:00:003
1297130994224,13cyclictest275-21dbus-daemon11:15:013
1297130994224,13cyclictest275-21dbus-daemon11:15:013
1297130994214,6cyclictest131rcu_preempt07:35:013
129711899425,4cyclictest131rcu_preempt11:15:021
129711899425,4cyclictest131rcu_preempt11:15:021
129711899424,4cyclictest131rcu_preempt10:05:331
129711899424,27cyclictest1298586-21kworker/u8:207:45:021
1297118994232,6cyclictest1302027-21latency_hist08:45:021
1297118994229,9cyclictest1392333-21cat12:20:011
129711899422,3cyclictest131rcu_preempt10:27:291
129711599427,5cyclictest131rcu_preempt08:30:030
129711599427,5cyclictest131rcu_preempt08:30:030
129711599421,9cyclictest1298117-21/usr/sbin/munin07:30:000
129713099413,3cyclictest131rcu_preempt11:30:513
1297130994128,10cyclictest1-21systemd08:15:003
1297130994124,12cyclictest37-21ksoftirqd/308:50:003
1297130994115,9cyclictest37-21ksoftirqd/309:26:213
129711899418,8cyclictest131rcu_preempt12:30:011
129711899413,3cyclictest131rcu_preempt07:40:001
1297118994129,10cyclictest23-21ksoftirqd/110:00:311
1297118994126,12cyclictest1299379-21sshd07:54:341
129711899412,3cyclictest131rcu_preempt09:49:041
129711899412,3cyclictest131rcu_preempt09:19:341
1297118994123,14cyclictest23-21ksoftirqd/112:10:171
129711899411,3cyclictest131rcu_preempt09:55:021
129711899411,3cyclictest131rcu_preempt09:05:011
129711599415,8cyclictest12-21ksoftirqd/007:15:010
129713099408,6cyclictest131rcu_preempt10:20:003
129713099403,3cyclictest131rcu_preempt11:30:023
1297130994030,7cyclictest3467602-21systemd-journal12:25:333
129713099402,5cyclictest131rcu_preempt09:00:003
1297130994025,10cyclictest0-21swapper/310:22:303
1297130994022,14cyclictest275-21dbus-daemon12:05:013
1297130994022,14cyclictest275-21dbus-daemon12:05:003
129711899404,3cyclictest131rcu_preempt09:27:131
1297118994030,6cyclictest1380569-21cron11:55:001
1297118994025,10cyclictest3467602-21systemd-journal08:09:341
1297118994020,13cyclictest1-21systemd08:40:001
129711899401,3cyclictest131rcu_preempt10:48:041
129711599409,6cyclictest131rcu_preempt07:35:020
129711599408,10cyclictest12-21ksoftirqd/007:43:570
1297115994014,21cyclictest151rcuc/007:25:020
1297115994013,11cyclictest12-21ksoftirqd/008:35:000
1297130993927,9cyclictest1-21systemd09:10:013
1297130993927,10cyclictest37-21ksoftirqd/311:40:023
1297130993926,9cyclictest309-21in:imuxsock11:40:023
1297130993925,9cyclictest1301138-21cat08:25:023
1297130993925,11cyclictest37-21ksoftirqd/307:25:023
1297130993910,19cyclictest131rcu_preempt10:00:033
1297118993930,6cyclictest1335932-21sh10:20:021
1297118993927,7cyclictest1354595-21sendmail10:59:591
1297118993925,10cyclictest23-21ksoftirqd/109:58:231
129711899392,3cyclictest131rcu_preempt10:15:031
1297118993923,12cyclictest3467602-21systemd-journal08:55:021
1297118993923,12cyclictest3467602-21systemd-journal08:55:011
1297130993827,9cyclictest37-21ksoftirqd/310:30:023
1297130993827,8cyclictest1322370-21sshd09:50:333
1297130993823,11cyclictest37-21ksoftirqd/309:40:033
129713099382,10cyclictest1393451-21ssh12:22:223
1297130993818,15cyclictest275-21dbus-daemon08:30:023
1297130993818,15cyclictest275-21dbus-daemon08:30:013
1297130993814,6cyclictest131rcu_preempt08:45:023
1297130993810,6cyclictest131rcu_preempt12:15:003
1297130993810,5cyclictest131rcu_preempt07:15:003
129711899383,3cyclictest131rcu_preempt09:00:001
1297118993828,7cyclictest23-21ksoftirqd/108:50:031
1297118993824,9cyclictest309-21in:imuxsock09:05:331
129711899382,3cyclictest131rcu_preempt07:30:021
1297118993823,11cyclictest3467602-21systemd-journal09:15:001
129711899381,3cyclictest131rcu_preempt09:23:251
129713099374,4cyclictest131rcu_preempt07:20:003
129713099373,5cyclictest131rcu_preempt08:40:013
1297130993725,10cyclictest37-21ksoftirqd/309:24:493
1297130993724,10cyclictest37-21ksoftirqd/308:20:013
1297130993717,9cyclictest37-21ksoftirqd/310:43:403
1297130993715,17cyclictest275-21dbus-daemon11:10:003
129711899373,6cyclictest1399292-21munin-run12:35:011
129711899373,3cyclictest131rcu_preempt08:35:001
1297118993727,7cyclictest1338376-21cat10:25:021
1297118993720,12cyclictest275-21dbus-daemon07:25:001
1297118993717,15cyclictest275-21dbus-daemon12:10:011
1297130993626,8cyclictest37-21ksoftirqd/309:30:353
1297130993619,12cyclictest3467602-21systemd-journal08:09:593
1297130993617,14cyclictest275-21dbus-daemon08:05:013
1297130993610,17cyclictest37-21ksoftirqd/307:30:013
129711899362,6cyclictest131rcu_preempt08:04:591
1297118993625,9cyclictest23-21ksoftirqd/111:23:381
1297118993624,9cyclictest23-21ksoftirqd/111:06:301
1297118993621,10cyclictest309-21in:imuxsock08:25:031
1297118993621,10cyclictest309-21in:imuxsock08:25:021
129713099358,5cyclictest131rcu_preempt08:50:323
129713099358,5cyclictest131rcu_preempt08:50:323
1297130993523,10cyclictest37-21ksoftirqd/307:50:003
1297130993522,10cyclictest0-21swapper/312:00:003
129713099352,13cyclictest1299410-21munin-run07:55:003
129711899352,3cyclictest131rcu_preempt08:20:001
1297118993522,10cyclictest23-21ksoftirqd/107:15:021
129713099342,21cyclictest0-21swapper/307:10:023
1297118993426,5cyclictest1297128-21ls07:10:011
1297118993422,9cyclictest23-21ksoftirqd/107:57:451
129713099331,12cyclictest1-21systemd08:35:003
1297118993318,12cyclictest3467602-21systemd-journal08:25:021
129713099321,5cyclictest131rcu_preempt07:45:013
1297118993018,9cyclictest1298936-21cat07:45:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional