You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-03 - 05:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Mon Aug 03, 2026 00:45:20)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1459221991253,5cyclictest131rcu_preempt22:40:011
1459227998714,63cyclictest131rcu_preempt23:40:013
145921999786,49cyclictest12-21ksoftirqd/023:15:010
145921999714,48cyclictest1553968-21ssh00:20:010
1459219997011,9cyclictest131rcu_preempt22:43:170
1459219996411,12cyclictest131rcu_preempt23:45:010
1459219996219,28cyclictest12-21ksoftirqd/021:37:580
1459219996124,33cyclictest12-21ksoftirqd/000:12:540
1459219996116,41cyclictest12-21ksoftirqd/022:35:440
1459219995913,24cyclictest131rcu_preempt23:30:030
145921999586,6cyclictest131rcu_preempt21:10:000
1459219995830,12cyclictest1504858-21ssh22:33:510
1459219995829,13cyclictest1561580-21ssh00:35:460
1459219995828,15cyclictest1511035-21ssh22:46:410
1459219995820,19cyclictest16850irq/134-800000000:35:010
1459219995817,37cyclictest12-21ksoftirqd/000:21:430
1459219995815,38cyclictest12-21ksoftirqd/022:03:040
1459219995731,11cyclictest1527254-21ssh23:21:260
145921999571,39cyclictest1468229-21ssh21:15:380
145921999566,45cyclictest1477191-21modprobe21:35:000
1459219995634,18cyclictest12-21ksoftirqd/022:26:440
1459219995627,13cyclictest1500643-21ssh22:24:560
1459219995627,13cyclictest1500643-21ssh22:24:560
1459219995625,15cyclictest1545339-21ssh00:01:300
145921999562,49cyclictest1488885-21cron22:00:010
1459219995620,7cyclictest12-21ksoftirqd/020:15:020
1459219995620,31cyclictest16850irq/134-800000022:09:450
1459227995510,6cyclictest131rcu_preempt22:20:013
1459224995546,6cyclictest1491305-21kworker/2:100:00:012
1459224995546,6cyclictest1491305-21kworker/2:100:00:002
1459219995541,10cyclictest12-21ksoftirqd/019:20:000
1459219995534,17cyclictest1484434-21latency_hist21:50:030
1459219995528,24cyclictest1466746-21ssh21:12:450
1459219995527,15cyclictest131rcu_preempt22:53:540
1459219995526,14cyclictest1516600-21ssh22:59:000
145921999552,29cyclictest131rcu_preempt21:40:530
1459219995519,32cyclictest16850irq/134-800000022:12:030
145921999551,5cyclictest131rcu_preempt23:20:010
1459219995510,37cyclictest12-21ksoftirqd/000:26:210
1459224995445,6cyclictest1464887-21kworker/2:221:50:572
1459219995425,14cyclictest1518583-21ssh23:02:580
1459219995418,32cyclictest16850irq/134-800000023:31:070
1459219995417,33cyclictest16850irq/134-800000023:49:400
145922199532,8cyclictest3467602-21systemd-journal23:45:001
145921999537,5cyclictest131rcu_preempt21:26:350
1459219995330,6cyclictest131rcu_preempt21:25:010
1459219995318,32cyclictest16850irq/134-800000021:50:430
1459219995317,32cyclictest16850irq/134-800000023:57:490
1459219995317,32cyclictest16850irq/134-800000023:57:490
1459219995317,31cyclictest16850irq/134-800000000:09:500
1459219995317,31cyclictest16850irq/134-800000000:09:490
1459219995316,32cyclictest16850irq/134-800000023:39:180
145922799528,6cyclictest131rcu_preempt00:15:003
1459224995244,5cyclictest1491305-21kworker/2:123:42:112
145922199526,7cyclictest131rcu_preempt21:25:001
1459219995215,32cyclictest16850irq/134-800000022:16:390
145922799518,6cyclictest131rcu_preempt19:45:003
145922799516,6cyclictest131rcu_preempt21:40:013
1459227995134,12cyclictest275-21dbus-daemon22:00:003
1459227995110,6cyclictest131rcu_preempt00:00:013
1459227995110,6cyclictest131rcu_preempt00:00:013
1459224995143,5cyclictest1545231-21kworker/2:000:38:402
1459224995143,5cyclictest1464887-21kworker/2:221:44:312
1459221995110,5cyclictest131rcu_preempt23:10:001
1459219995118,29cyclictest16850irq/134-800000023:50:410
1459219995110,5cyclictest131rcu_preempt20:05:010
1459227995031,13cyclictest275-21dbus-daemon22:55:013
1459224995042,5cyclictest1503350-21kworker/2:223:04:332
1459224995042,5cyclictest1491305-21kworker/2:123:46:132
1459224995042,5cyclictest1460989-21kworker/2:021:28:322
1459224995040,7cyclictest1491305-21kworker/2:122:26:492
1459221995017,22cyclictest23-21ksoftirqd/121:50:021
145921999503,11cyclictest1460708-21munin-run19:40:000
145922799499,7cyclictest131rcu_preempt20:45:033
145922799497,9cyclictest37-21ksoftirqd/320:35:023
145922799496,6cyclictest131rcu_preempt00:35:003
1459227994931,13cyclictest275-21dbus-daemon00:40:023
1459227994919,7cyclictest131rcu_preempt00:10:013
1459227994919,7cyclictest131rcu_preempt00:10:003
1459224994941,5cyclictest1503350-21kworker/2:223:11:002
1459224994941,5cyclictest1503350-21kworker/2:222:55:552
1459224994941,5cyclictest1491305-21kworker/2:122:46:052
1459224994941,5cyclictest1491305-21kworker/2:100:13:422
1459224994941,5cyclictest1484250-21kworker/2:322:01:402
1459224994940,6cyclictest1491305-21kworker/2:122:17:052
1459224994940,6cyclictest1452155-21kworker/2:319:23:442
1459224994940,6cyclictest1452155-21kworker/2:319:23:442
145922499492,15cyclictest1483867-21ssh21:49:062
145921999498,7cyclictest131rcu_preempt19:45:000
145921999493,15cyclictest1461769-21true20:00:000
1459219994910,5cyclictest131rcu_preempt20:35:020
1459227994816,20cyclictest131rcu_preempt23:00:013
1459224994840,5cyclictest1545231-21kworker/2:000:33:542
1459224994840,5cyclictest1491305-21kworker/2:123:32:582
1459224994840,5cyclictest1491305-21kworker/2:122:53:172
1459224994840,5cyclictest1460989-21kworker/2:021:11:422
1459224994839,6cyclictest1464887-21kworker/2:221:22:062
1459224994833,12cyclictest30-21ksoftirqd/200:05:012
145922499482,3cyclictest131rcu_preempt20:10:022
145922499481,4cyclictest131rcu_preempt21:10:012
145922499481,3cyclictest131rcu_preempt00:07:472
145922499481,3cyclictest131rcu_preempt00:07:462
145922199488,5cyclictest131rcu_preempt20:33:551
1459219994814,29cyclictest16850irq/134-800000023:09:280
1459219994810,6cyclictest131rcu_preempt19:25:000
1459219994810,6cyclictest131rcu_preempt19:25:000
145922799479,7cyclictest131rcu_preempt22:45:013
145922799479,6cyclictest131rcu_preempt22:33:143
145922799479,6cyclictest131rcu_preempt21:26:053
145922799479,5cyclictest131rcu_preempt23:50:023
145922799479,5cyclictest131rcu_preempt00:19:103
145922799477,5cyclictest131rcu_preempt21:55:003
145922799472,5cyclictest131rcu_preempt00:03:113
145922799471,6cyclictest131rcu_preempt23:15:013
1459227994716,7cyclictest131rcu_preempt20:00:003
145922799471,5cyclictest131rcu_preempt23:26:543
1459227994712,6cyclictest131rcu_preempt19:20:023
1459227994711,6cyclictest131rcu_preempt19:35:003
1459224994740,4cyclictest1491305-21kworker/2:100:27:142
1459224994739,5cyclictest1503350-21kworker/2:223:07:182
1459224994739,5cyclictest1491305-21kworker/2:122:10:292
1459224994739,5cyclictest1491305-21kworker/2:122:05:492
1459224994738,5cyclictest1452155-21kworker/2:319:30:002
1459224994731,12cyclictest3467602-21systemd-journal20:20:002
1459224994731,11cyclictest1526435-21systemctl23:19:592
145922499471,3cyclictest131rcu_preempt21:31:102
145922499471,3cyclictest131rcu_preempt21:18:252
145921999479,6cyclictest131rcu_preempt19:50:000
1459219994719,24cyclictest1460193-21latency_hist19:30:020
145922799469,6cyclictest131rcu_preempt21:00:013
145922799468,6cyclictest131rcu_preempt21:35:013
145922799468,5cyclictest131rcu_preempt22:06:273
145922799462,4cyclictest131rcu_preempt19:40:003
1459227994614,6cyclictest1509939-21kworker/u8:323:42:293
1459227994614,6cyclictest131rcu_preempt21:45:023
1459227994611,8cyclictest131rcu_preempt21:10:523
1459224994638,5cyclictest1496002-21kworker/2:022:23:412
1459224994638,5cyclictest1496002-21kworker/2:022:23:402
1459224994638,5cyclictest1491305-21kworker/2:123:29:392
1459224994638,5cyclictest1484250-21kworker/2:321:57:372
145922499462,14cyclictest1541089-21ssh23:50:452
145922199465,6cyclictest131rcu_preempt22:31:541
1459221994634,9cyclictest1543930-21sshd23:56:541
1459221994634,9cyclictest1543930-21sshd23:56:541
1459221994612,6cyclictest131rcu_preempt22:26:541
145921999462,8cyclictest12-21ksoftirqd/021:05:000
145922799458,5cyclictest131rcu_preempt21:20:003
145922799458,4cyclictest131rcu_preempt23:07:203
145922799455,7cyclictest131rcu_preempt23:20:013
145922799454,5cyclictest131rcu_preempt20:20:013
145922799454,4cyclictest131rcu_preempt20:30:013
1459227994529,12cyclictest37-21ksoftirqd/323:50:143
1459227994525,15cyclictest275-21dbus-daemon20:05:013
1459227994516,12cyclictest37-21ksoftirqd/323:25:003
145922799451,3cyclictest131rcu_preempt22:25:033
145922799451,3cyclictest131rcu_preempt22:25:023
1459227994510,5cyclictest131rcu_preempt22:26:173
1459224994539,4cyclictest1491305-21kworker/2:100:22:192
1459224994536,5cyclictest1491305-21kworker/2:122:41:032
145922499451,3cyclictest131rcu_preempt22:30:362
145922499451,3cyclictest131rcu_preempt00:19:012
145922199455,5cyclictest131rcu_preempt23:28:331
1459221994531,9cyclictest23-21ksoftirqd/120:20:521
1459221994531,11cyclictest1535837-21systemctl23:39:591
1459221994526,14cyclictest275-21dbus-daemon00:40:021
145922199452,3cyclictest150-21pr/ttyS223:52:261
1459219994510,5cyclictest131rcu_preempt20:23:540
145922799449,6cyclictest131rcu_preempt21:47:373
145922799442,5cyclictest131rcu_preempt23:35:023
145922799441,4cyclictest131rcu_preempt21:25:013
1459227994410,6cyclictest131rcu_preempt22:14:553
1459227994410,5cyclictest131rcu_preempt20:25:013
1459224994436,5cyclictest1461505-21kworker/2:120:38:272
1459224994435,6cyclictest1452155-21kworker/2:319:50:012
145922499443,3cyclictest131rcu_preempt22:39:552
1459224994431,9cyclictest1-21systemd21:05:012
1459224994430,11cyclictest1-21systemd20:05:012
1459224994430,10cyclictest3467602-21systemd-journal19:13:532
145922199448,5cyclictest131rcu_preempt20:10:021
145922199447,7cyclictest23-21ksoftirqd/120:45:021
145922199444,7cyclictest23-21ksoftirqd/121:13:231
145922199444,7cyclictest23-21ksoftirqd/121:00:031
1459221994430,10cyclictest275-21dbus-daemon20:00:011
1459221994428,12cyclictest1517123-21basename23:00:001
1459221994420,7cyclictest221rcuc/119:50:011
145922199441,9cyclictest23-21ksoftirqd/100:20:011
1459219994428,13cyclictest1464576-21sshd20:54:550
1459219994425,15cyclictest12-21ksoftirqd/020:40:020
1459219994423,16cyclictest3467602-21systemd-journal19:35:000
145922799438,5cyclictest131rcu_preempt00:23:543
145922799434,6cyclictest131rcu_preempt22:40:003
145922799434,4cyclictest131rcu_preempt23:05:023
1459227994324,13cyclictest3467602-21systemd-journal20:40:003
145922799431,4cyclictest131rcu_preempt22:46:263
145922799431,4cyclictest131rcu_preempt20:15:013
145922499437,7cyclictest131rcu_preempt21:40:002
145922499436,7cyclictest131rcu_preempt23:35:522
145922499433,3cyclictest131rcu_preempt23:21:362
1459224994329,11cyclictest1463165-21sed20:25:022
145922199435,7cyclictest23-21ksoftirqd/122:52:581
145922199434,3cyclictest131rcu_preempt00:33:331
1459221994328,11cyclictest1-21systemd22:00:011
1459221994326,12cyclictest1489716-21sshd22:01:141
145922199431,6cyclictest1496628-21ssh22:16:071
145921999438,6cyclictest131rcu_preempt20:50:030
1459219994319,5cyclictest12-21ksoftirqd/020:10:010
1459219994310,5cyclictest131rcu_preempt19:55:010
145922799424,5cyclictest131rcu_preempt22:05:013
1459227994227,11cyclictest3467602-21systemd-journal19:50:023
1459227994210,5cyclictest131rcu_preempt21:10:033
145922499424,3cyclictest131rcu_preempt19:55:002
1459224994232,6cyclictest1452155-21kworker/2:319:45:022
1459224994225,12cyclictest275-21dbus-daemon20:35:012
145922199429,6cyclictest131rcu_preempt21:35:391
145922199426,7cyclictest23-21ksoftirqd/100:23:521
145922199425,6cyclictest23-21ksoftirqd/122:45:391
145922199423,3cyclictest131rcu_preempt23:47:331
1459221994228,10cyclictest275-21dbus-daemon19:55:021
1459221994224,13cyclictest275-21dbus-daemon20:40:001
1459221994221,6cyclictest23-21ksoftirqd/123:13:131
145921999426,5cyclictest131rcu_preempt19:10:020
145921999422,8cyclictest1464835-21cron21:00:000
145922799419,7cyclictest131rcu_preempt19:50:013
145922799414,10cyclictest37-21ksoftirqd/320:55:013
145922799412,5cyclictest131rcu_preempt19:15:023
1459227994125,11cyclictest37-21ksoftirqd/300:28:223
1459224994128,9cyclictest30-21ksoftirqd/220:30:012
1459224994124,14cyclictest275-21dbus-daemon19:40:012
145922199419,5cyclictest131rcu_preempt21:43:231
145922199416,3cyclictest131rcu_preempt23:03:001
145922199413,4cyclictest131rcu_preempt00:26:191
1459221994129,9cyclictest1468766-21sshd21:16:541
1459221994126,12cyclictest1474844-21pluginstate21:30:021
1459221994123,14cyclictest3467602-21systemd-journal21:10:011
1459221994115,8cyclictest131rcu_preempt22:25:021
1459221994115,8cyclictest131rcu_preempt22:25:021
1459219994132,6cyclictest1715541-1kworker/0:0H20:35:330
145922799409,5cyclictest131rcu_preempt19:25:013
145922799409,5cyclictest131rcu_preempt19:25:013
1459227994015,6cyclictest131rcu_preempt20:09:543
1459224994032,5cyclictest1461505-21kworker/2:120:45:192
145922199407,4cyclictest131rcu_preempt21:55:001
145922199405,4cyclictest131rcu_preempt21:32:351
145922199404,3cyclictest131rcu_preempt19:25:001
145922199404,3cyclictest131rcu_preempt19:25:001
145922199403,3cyclictest131rcu_preempt19:30:021
1459221994028,9cyclictest23-21ksoftirqd/123:20:121
1459221994028,9cyclictest23-21ksoftirqd/100:10:011
1459221994028,9cyclictest23-21ksoftirqd/100:10:011
1459221994026,10cyclictest3467602-21systemd-journal19:20:001
1459221994026,10cyclictest1496039-21latency_hist22:15:011
145922199401,5cyclictest0-21swapper/122:07:501
145922799396,4cyclictest131rcu_preempt21:05:013
145922499393,3cyclictest131rcu_preempt19:20:022
1459224993931,5cyclictest1461505-21kworker/2:120:45:022
1459224993923,12cyclictest1464564-21sshd20:51:542
1459224993923,12cyclictest1-21systemd20:00:012
1459224993923,11cyclictest30-21ksoftirqd/219:35:002
145922199398,3cyclictest131rcu_preempt20:15:001
145922199393,4cyclictest131rcu_preempt00:11:431
145922199392,22cyclictest0-21swapper/100:02:521
1459221993921,14cyclictest1-21systemd20:20:021
1459221993917,5cyclictest23-21ksoftirqd/120:50:011
145921999394,6cyclictest131rcu_preempt19:14:530
145922799385,6cyclictest131rcu_preempt19:30:003
145922799381,5cyclictest131rcu_preempt20:50:003
145922499383,3cyclictest131rcu_preempt20:55:592
1459224993827,8cyclictest1462502-21cron20:15:002
145922199384,5cyclictest131rcu_preempt20:05:011
145922199384,3cyclictest131rcu_preempt22:43:191
145922199384,3cyclictest131rcu_preempt20:30:001
145922199382,4cyclictest131rcu_preempt23:17:001
1459221993820,13cyclictest1-21systemd20:55:011
145921999386,5cyclictest131rcu_preempt20:30:020
145921999385,5cyclictest131rcu_preempt20:19:540
145922199374,3cyclictest131rcu_preempt19:15:021
1459221993729,6cyclictest23-21ksoftirqd/119:45:011
1459221993725,8cyclictest3467602-21systemd-journal19:35:011
145922199371,4cyclictest131rcu_preempt19:40:011
145922199371,3cyclictest131rcu_preempt23:30:121
145922199365,3cyclictest131rcu_preempt19:10:021
145922499353,3cyclictest131rcu_preempt19:10:022
1459227993423,8cyclictest1459360-21sed19:10:023
145922199342,3cyclictest131rcu_preempt21:00:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional