You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-30 - 12:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sat May 30, 2026 00:45:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1438539999630,34cyclictest37-21ksoftirqd/321:28:383
1438539999626,33cyclictest1457492-21sh21:38:443
143852599932,64cyclictest1469335-21rm22:05:020
143853999813,43cyclictest1528421-21ssh00:17:013
1438539997422,46cyclictest275-21dbus-daemon21:55:013
1438539997036,26cyclictest275-21dbus-daemon00:00:003
1438525996928,35cyclictest275-21dbus-daemon22:15:000
143853999682,20cyclictest1537756-21ssh00:38:033
143853999671,22cyclictest1448342-21ssh21:18:033
143853999671,22cyclictest1448342-21ssh21:18:033
143853999662,19cyclictest1508556-21ssh23:32:463
1438530996614,8cyclictest131rcu_preempt19:10:011
143853999652,58cyclictest1472181-21ssh22:11:103
143853999652,19cyclictest1534510-21ssh00:30:253
143853999652,19cyclictest1534510-21ssh00:30:253
1438539996521,40cyclictest1442390-21sh20:25:013
1438539996521,38cyclictest309-21in:imuxsock23:42:223
143853999651,16cyclictest1459481-21ssh21:43:033
1438530996520,19cyclictest0-21swapper/122:25:221
143853999642,17cyclictest1527334-21ssh00:14:533
143853999642,17cyclictest1527334-21ssh00:14:523
143853999642,15cyclictest1477430-21ssh22:23:063
143853999642,14cyclictest1480144-21sshd22:29:223
143853999641,21cyclictest1523769-21ssh00:06:393
143853999641,16cyclictest1503086-21ssh23:20:193
1438539996331,22cyclictest3467602-21systemd-journal22:00:003
143853999632,8cyclictest1534114-21munin-run00:30:013
143853999631,8cyclictest1483893-21ssh22:37:353
143853999631,12cyclictest364-21snmpd20:56:083
1438534996313,12cyclictest30-21ksoftirqd/222:30:022
1438530996318,11cyclictest23-21ksoftirqd/121:55:011
1438539996230,27cyclictest3467602-21systemd-journal20:10:003
143853999622,18cyclictest1481682-21ssh22:32:363
143853999622,15cyclictest1439027-21munin-run19:20:023
143853999622,14cyclictest1522865-21ssh00:04:573
143853999621,7cyclictest1444417-21sshd21:00:223
143853999621,7cyclictest1443872-21cat20:50:033
143853999621,5cyclictest1439275-21munin-run19:25:013
143853999621,11cyclictest48-21kcompactd023:35:283
143853999621,11cyclictest48-21kcompactd023:35:283
143853999621,11cyclictest1442849-21sshd20:30:223
143853999613,52cyclictest1497382-21ssh23:07:473
143853999613,52cyclictest1497382-21ssh23:07:473
143853999612,53cyclictest1500491-21ssh23:14:583
143853999612,17cyclictest1494912-21ssh23:02:153
143853999612,13cyclictest1500916-21ssh23:15:363
143853999611,9cyclictest1467872-21ssh22:01:343
143853999611,16cyclictest1462192-21sh21:49:203
143853999611,14cyclictest1530501-21ssh00:21:393
143853999611,13cyclictest1505361-21ssh23:25:263
143853999611,13cyclictest1505361-21ssh23:25:263
1438534996146,11cyclictest1-21systemd00:35:012
1438534996146,11cyclictest1-21systemd00:35:012
1438525996116,12cyclictest1518570-21ssh23:55:020
143853999602,17cyclictest1454618-21ssh21:32:103
143853999601,8cyclictest1489784-21ssh22:50:313
143853999601,8cyclictest1475143-21ssh22:18:003
143853999601,7cyclictest1470543-21ssh22:07:373
143853999601,12cyclictest1449904-21ssh21:21:283
1438525996014,7cyclictest12-21ksoftirqd/021:50:470
143853999594,47cyclictest1439778-21true19:35:013
143853999592,50cyclictest1515242-21ssh23:47:353
143853999592,17cyclictest1493462-21ssh22:59:083
143853999592,17cyclictest1493462-21ssh22:59:083
143853999592,16cyclictest1486196-21ssh22:42:473
143853999591,6cyclictest1440569-21munin-run19:50:013
143853999591,4cyclictest364-21snmpd20:00:563
143853999591,13cyclictest1516543-21ssh23:50:213
143853999591,11cyclictest1487725-21sh22:46:033
143853999591,11cyclictest1487725-21sh22:46:033
143853499595,18cyclictest30-21ksoftirqd/220:20:012
1438534995940,12cyclictest1444708-21diskmemload23:10:032
143852599597,33cyclictest131rcu_preempt00:09:520
1438525995915,39cyclictest12-21ksoftirqd/000:00:010
143853999584,47cyclictest1442090-21cron20:20:003
143853999582,4cyclictest364-21snmpd19:55:553
143853999581,7cyclictest1445737-21ssh21:12:053
1438534995813,5cyclictest131rcu_preempt23:45:012
143852599589,5cyclictest131rcu_preempt21:29:460
143852599588,31cyclictest131rcu_preempt23:12:480
143852599586,33cyclictest12-21ksoftirqd/000:19:480
1438525995848,6cyclictest1537962-21sh00:38:300
1438525995817,37cyclictest12-21ksoftirqd/022:40:290
143853999573,46cyclictest275-21dbus-daemon20:45:003
143853999572,7cyclictest1443908-21cron20:55:003
143853999572,7cyclictest1443149-21munin-run20:40:013
143853999572,49cyclictest1440305-21munin-run19:45:013
1438525995716,35cyclictest16850irq/134-800000022:00:000
143852599571,53cyclictest0-21swapper/019:15:020
143853999562,48cyclictest1444828-21wc21:10:023
143853999561,5cyclictest364-21snmpd20:26:003
143853499563,12cyclictest30-21ksoftirqd/220:39:592
1438534995615,6cyclictest131rcu_preempt19:10:002
1438525995619,33cyclictest16850irq/134-800000022:05:230
1438525995617,23cyclictest16850irq/134-800000023:06:350
1438525995617,23cyclictest16850irq/134-800000023:06:350
143853999553,44cyclictest1440945-21cat19:55:023
143853499551,22cyclictest0-21swapper/222:37:392
143853099552,18cyclictest1520593-40mandb00:00:011
1438525995528,12cyclictest1515590-21ssh23:48:310
1438525995519,32cyclictest12-21ksoftirqd/000:03:400
1438525995518,32cyclictest16850irq/134-800000023:40:030
143853999543,44cyclictest309-21in:imuxsock19:15:003
143853999541,7cyclictest364-21snmpd20:14:413
143853999541,7cyclictest364-21snmpd20:14:413
1438534995436,13cyclictest275-21dbus-daemon00:05:012
1438534995410,5cyclictest131rcu_preempt21:47:292
1438525995427,23cyclictest1483979-21ssh22:37:470
1438525995424,26cyclictest12-21ksoftirqd/000:11:460
1438525995424,26cyclictest12-21ksoftirqd/000:11:460
1438525995418,32cyclictest16850irq/134-800000022:47:460
1438525995418,32cyclictest16850irq/134-800000022:47:460
1438525995418,32cyclictest16850irq/134-800000022:28:150
1438525995418,32cyclictest16850irq/134-800000022:22:350
143852599541,7cyclictest275-21dbus-daemon21:35:010
1438525995417,32cyclictest16850irq/134-800000023:36:510
1438525995417,32cyclictest16850irq/134-800000023:36:510
1438525995417,32cyclictest16850irq/134-800000000:26:510
143853499539,5cyclictest131rcu_preempt21:10:012
143853499538,12cyclictest30-21ksoftirqd/221:30:022
1438534995315,11cyclictest30-21ksoftirqd/223:08:402
1438534995315,11cyclictest30-21ksoftirqd/223:08:402
143853099536,8cyclictest23-21ksoftirqd/119:30:001
1438530995315,6cyclictest131rcu_preempt22:36:121
1438530995310,5cyclictest131rcu_preempt21:59:061
1438525995319,30cyclictest16850irq/134-800000021:42:080
1438525995319,29cyclictest16850irq/134-800000022:32:550
1438525995318,31cyclictest16850irq/134-800000023:31:000
1438525995317,31cyclictest16850irq/134-800000021:17:460
1438525995317,31cyclictest16850irq/134-800000021:17:450
1438525995316,32cyclictest16850irq/134-800000023:25:250
1438525995316,32cyclictest16850irq/134-800000023:25:250
143853999523,44cyclictest0-21swapper/319:27:103
1438534995241,7cyclictest291rcuc/222:48:262
1438534995241,7cyclictest291rcuc/222:48:262
143853499524,11cyclictest30-21ksoftirqd/221:52:252
143853499522,14cyclictest30-21ksoftirqd/220:00:002
1438534995217,6cyclictest131rcu_preempt21:22:012
143853099528,5cyclictest131rcu_preempt00:20:181
1438525995218,30cyclictest16850irq/134-800000021:20:240
1438525995217,31cyclictest16850irq/134-800000021:13:400
1438525995217,31cyclictest16850irq/134-800000000:20:060
1438525995216,32cyclictest16850irq/134-800000023:20:190
1438525995216,32cyclictest16850irq/134-800000000:31:120
1438525995216,32cyclictest16850irq/134-800000000:31:120
143853499516,9cyclictest30-21ksoftirqd/223:18:452
143853499516,7cyclictest291rcuc/222:52:242
1438534995143,5cyclictest1530017-21kworker/2:200:37:302
1438534995143,5cyclictest1480773-21kworker/2:023:59:352
1438534995143,5cyclictest1441369-21kworker/2:123:26:592
1438534995143,5cyclictest1441369-21kworker/2:123:26:592
1438534995143,5cyclictest1441369-21kworker/2:121:10:102
1438534995134,12cyclictest30-21ksoftirqd/223:50:302
143853499512,8cyclictest30-21ksoftirqd/221:37:542
143853499512,12cyclictest30-21ksoftirqd/223:48:222
1438534995116,6cyclictest131rcu_preempt00:19:512
1438534995113,6cyclictest131rcu_preempt23:05:002
1438530995132,7cyclictest23-21ksoftirqd/123:36:511
1438530995132,7cyclictest23-21ksoftirqd/123:36:501
1438525995133,14cyclictest1492027-21ssh22:55:290
1438525995133,14cyclictest1492027-21ssh22:55:290
1438525995116,31cyclictest16850irq/134-800000022:17:330
1438525995115,32cyclictest16850irq/134-800000021:35:500
1438525995115,31cyclictest16850irq/134-800000021:47:050
143853999505,39cyclictest3467602-21systemd-journal19:39:223
1438534995043,4cyclictest1441369-21kworker/2:121:18:592
1438534995043,4cyclictest1441369-21kworker/2:121:18:592
1438534995042,5cyclictest1480773-21kworker/2:023:37:422
1438534995042,5cyclictest1480773-21kworker/2:023:37:422
1438534995042,5cyclictest1480773-21kworker/2:022:56:342
1438534995042,5cyclictest1480773-21kworker/2:022:56:342
1438534995042,5cyclictest1443428-21kworker/2:021:57:442
1438534995042,5cyclictest1441369-21kworker/2:122:34:022
1438534995041,6cyclictest1441369-21kworker/2:122:22:052
1438534995041,5cyclictest1443428-21kworker/2:021:42:132
1438534995036,10cyclictest1-21systemd22:10:012
143853099509,5cyclictest131rcu_preempt23:52:111
143853099501,5cyclictest131rcu_preempt23:25:011
1438530995010,5cyclictest131rcu_preempt21:16:391
1438530995010,5cyclictest131rcu_preempt21:16:391
1438530995010,5cyclictest131rcu_preempt00:20:001
143852599501,5cyclictest131rcu_preempt23:05:000
143852599501,5cyclictest131rcu_preempt20:00:000
1438525995015,31cyclictest16850irq/134-800000022:51:540
143853499499,5cyclictest131rcu_preempt22:00:362
1438534994941,5cyclictest1520594-21kworker/2:100:06:222
1438534994941,5cyclictest1480773-21kworker/2:023:30:262
1438534994941,5cyclictest1443428-21kworker/2:022:15:192
1438534994941,5cyclictest1443428-21kworker/2:022:12:052
1438534994941,5cyclictest1441369-21kworker/2:120:45:082
1438534994941,5cyclictest1432237-21kworker/2:320:40:242
1438534994941,5cyclictest1432237-21kworker/2:320:32:232
1438534994941,5cyclictest1432237-21kworker/2:320:06:222
1438534994941,5cyclictest1432237-21kworker/2:319:48:272
1438534994940,6cyclictest1480773-21kworker/2:022:44:562
1438534994940,6cyclictest1432237-21kworker/2:320:25:142
143853499492,15cyclictest30-21ksoftirqd/200:25:552
143853099499,5cyclictest131rcu_preempt22:30:481
143853099497,4cyclictest131rcu_preempt21:45:421
1438530994924,9cyclictest23-21ksoftirqd/100:05:321
1438530994921,11cyclictest23-21ksoftirqd/123:18:591
1438530994911,6cyclictest131rcu_preempt23:45:011
1438530994910,5cyclictest131rcu_preempt21:26:031
1438530994910,5cyclictest131rcu_preempt20:37:221
143852599493,7cyclictest131rcu_preempt20:40:000
1438525994933,12cyclictest3467602-21systemd-journal20:55:010
143853999486,36cyclictest1438608-21cat19:10:023
1438534994841,4cyclictest1441369-21kworker/2:120:50:412
1438534994840,5cyclictest1520594-21kworker/2:100:10:362
1438534994840,5cyclictest1520594-21kworker/2:100:10:362
1438534994840,5cyclictest1480773-21kworker/2:023:20:052
1438534994840,5cyclictest1480773-21kworker/2:000:20:102
1438534994840,5cyclictest1441369-21kworker/2:121:32:272
1438534994840,5cyclictest1432237-21kworker/2:319:40:102
1438534994840,5cyclictest1432237-21kworker/2:319:17:082
143853099484,9cyclictest23-21ksoftirqd/121:36:061
143853099483,5cyclictest131rcu_preempt21:35:011
1438530994829,12cyclictest1-21systemd22:25:011
1438530994811,6cyclictest131rcu_preempt21:11:061
143852599482,7cyclictest12-21ksoftirqd/023:19:350
1438534994740,4cyclictest1432237-21kworker/2:320:23:042
1438534994740,4cyclictest1432237-21kworker/2:320:10:452
1438534994740,4cyclictest1432237-21kworker/2:320:10:452
1438534994740,4cyclictest1432237-21kworker/2:319:36:412
1438534994739,5cyclictest1441369-21kworker/2:121:00:592
1438534994739,5cyclictest1441369-21kworker/2:120:55:172
1438534994739,5cyclictest1432237-21kworker/2:320:00:552
1438534994739,5cyclictest1432237-21kworker/2:319:51:052
1438534994739,5cyclictest1432237-21kworker/2:319:33:292
1438534994739,5cyclictest1432237-21kworker/2:319:27:292
1438534994739,5cyclictest1432237-21kworker/2:319:23:392
1438534994739,5cyclictest1432237-21kworker/2:319:11:362
143853099476,4cyclictest131rcu_preempt23:33:111
143853099474,9cyclictest23-21ksoftirqd/123:05:001
143853099474,6cyclictest131rcu_preempt21:25:011
143853099471,5cyclictest131rcu_preempt22:06:221
143853099471,5cyclictest131rcu_preempt21:40:101
1438530994713,5cyclictest131rcu_preempt00:38:551
143853099469,5cyclictest131rcu_preempt22:01:291
143853099467,7cyclictest23-21ksoftirqd/123:12:251
143853099464,5cyclictest131rcu_preempt00:05:011
1438530994635,9cyclictest23-21ksoftirqd/100:28:141
1438530994628,14cyclictest1-21systemd19:50:001
143853099462,6cyclictest131rcu_preempt19:20:011
143853099462,5cyclictest131rcu_preempt22:56:331
143853099462,5cyclictest131rcu_preempt22:56:331
143853099461,5cyclictest1443883-21modprobe20:50:031
1438530994612,5cyclictest131rcu_preempt22:46:261
1438530994612,5cyclictest131rcu_preempt22:46:261
1438530994611,5cyclictest131rcu_preempt21:00:011
143853099452,5cyclictest131rcu_preempt00:10:261
143853099452,5cyclictest131rcu_preempt00:10:261
143853099452,4cyclictest131rcu_preempt22:15:001
143853099451,9cyclictest23-21ksoftirqd/123:45:341
1438530994510,5cyclictest131rcu_preempt22:51:201
1438525994538,4cyclictest16950irq/224-800000019:35:010
1438525994530,12cyclictest12-21ksoftirqd/020:10:020
143853099441,6cyclictest131rcu_preempt22:41:051
143853099441,4cyclictest131rcu_preempt20:50:011
1438530994411,5cyclictest131rcu_preempt23:06:181
1438530994411,5cyclictest131rcu_preempt23:06:181
1438530994411,5cyclictest131rcu_preempt00:31:371
1438530994411,5cyclictest131rcu_preempt00:31:371
143853099438,6cyclictest131rcu_preempt19:59:591
143853099437,5cyclictest131rcu_preempt20:05:231
143853099434,5cyclictest131rcu_preempt23:26:021
143853099434,5cyclictest131rcu_preempt23:26:021
143853099431,5cyclictest131rcu_preempt22:16:291
1438525994331,9cyclictest1444211-21systemd-run21:00:010
1438525994319,15cyclictest12-21ksoftirqd/020:27:310
143853099423,6cyclictest131rcu_preempt20:30:011
1438530994210,5cyclictest131rcu_preempt20:45:001
1438525994235,4cyclictest16950irq/224-800000021:06:220
1438525994228,10cyclictest1439494-21sshd19:27:220
143852599422,5cyclictest131rcu_preempt19:39:590
1438525994224,10cyclictest12-21ksoftirqd/020:44:230
143853099417,5cyclictest131rcu_preempt19:41:221
143853099416,5cyclictest131rcu_preempt20:02:221
143853099415,7cyclictest23-21ksoftirqd/120:15:211
1438530994130,8cyclictest1439242-21sshd19:23:221
1438530994127,11cyclictest275-21dbus-daemon21:10:011
1438530994123,13cyclictest275-21dbus-daemon20:25:011
1438525994134,4cyclictest16950irq/224-800000020:20:010
1438525994129,9cyclictest12-21ksoftirqd/019:45:020
1438525994126,8cyclictest1444456-21cut21:05:000
1438525994125,13cyclictest275-21dbus-daemon20:15:010
1438525994125,13cyclictest275-21dbus-daemon20:15:010
1438525994117,20cyclictest12-21ksoftirqd/020:50:020
143853099407,6cyclictest131rcu_preempt21:04:271
1438530994030,7cyclictest1441808-21sshd20:11:221
1438530994030,7cyclictest1441808-21sshd20:11:221
1438530994024,12cyclictest1439783-21systemd-run19:35:011
143853099401,4cyclictest131rcu_preempt19:53:211
143852599402,7cyclictest284-21systemd-logind19:25:000
143852599393,6cyclictest1438916-21sed19:15:020
1438525993925,10cyclictest275-21dbus-daemon20:25:010
143853099384,4cyclictest131rcu_preempt19:12:141
143853099383,8cyclictest23-21ksoftirqd/120:35:011
143852599389,16cyclictest1438527-21latency_hist19:10:000
1438525993820,14cyclictest1441308-21sshd20:01:500
1438530993726,8cyclictest23-21ksoftirqd/119:40:001
1438525993710,5cyclictest131rcu_preempt19:47:220
1438525993525,7cyclictest12-21ksoftirqd/020:31:440
143852599351,30cyclictest0-21swapper/019:52:270
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional