You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-14 - 21:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Sat Mar 14, 2026 12:45:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25510289910517,5cyclictest131rcu_preempt12:40:031
25510249910091,3cyclictest0-21swapper/009:25:020
2551024999288,2cyclictest0-21swapper/009:50:020
2551024999140,39cyclictest0-21swapper/010:25:010
2551024999140,39cyclictest0-21swapper/010:25:010
2551024997427,23cyclictest12-21ksoftirqd/010:45:010
2551024997254,13cyclictest0-21swapper/007:10:010
2551024997128,11cyclictest2628322-21ssh11:49:100
2551024997113,32cyclictest16850irq/134-800000012:15:070
255102499701,38cyclictest131rcu_preempt09:43:040
2551024996928,32cyclictest12-21ksoftirqd/011:25:010
2551024996924,13cyclictest2613806-21rm11:16:320
2551028996818,11cyclictest23-21ksoftirqd/110:35:021
2551024996829,17cyclictest12-21ksoftirqd/012:35:030
2551032996545,16cyclictest3467602-21systemd-journal10:00:002
2551028996323,7cyclictest23-21ksoftirqd/109:35:001
2551028996222,8cyclictest23-21ksoftirqd/110:40:021
2551028996213,6cyclictest131rcu_preempt07:40:001
2551024996228,23cyclictest12-21ksoftirqd/010:30:000
2551024996226,9cyclictest12-21ksoftirqd/012:05:110
2551024996130,22cyclictest12-21ksoftirqd/009:20:010
2551024996122,20cyclictest2588471-21ssh10:19:590
2551024996115,35cyclictest12-21ksoftirqd/012:15:010
2551036996028,18cyclictest37-21ksoftirqd/311:04:243
2551036996028,18cyclictest37-21ksoftirqd/311:04:243
2551028996020,5cyclictest131rcu_preempt11:52:431
2551024996028,16cyclictest2631951-21ssh11:57:080
2551024996028,16cyclictest2631951-21ssh11:57:080
2551028995918,12cyclictest23-21ksoftirqd/111:30:021
2551024995930,12cyclictest2623406-21ssh11:38:100
2551024995928,15cyclictest2601790-21ssh10:49:540
2551024995912,42cyclictest16850irq/134-800000011:15:010
2551024995912,42cyclictest16850irq/134-800000011:15:000
2551032995847,6cyclictest2572965-21kworker/2:011:51:322
2551028995822,9cyclictest23-21ksoftirqd/112:15:001
2551024995830,20cyclictest12-21ksoftirqd/011:35:000
2551028995716,6cyclictest131rcu_preempt09:15:001
2551024995740,14cyclictest16950irq/224-800000007:40:020
2551024995724,22cyclictest12-21ksoftirqd/011:52:200
2551028995615,5cyclictest131rcu_preempt09:59:231
2551028995610,6cyclictest131rcu_preempt11:33:231
2551024995638,15cyclictest16850irq/134-800000009:32:190
2551032995512,5cyclictest131rcu_preempt09:10:032
2551032995512,5cyclictest131rcu_preempt09:10:022
255102499557,3cyclictest0-21swapper/010:02:030
255102499557,3cyclictest0-21swapper/010:02:020
2551024995527,13cyclictest2583769-21ssh10:09:140
255102499551,38cyclictest131rcu_preempt12:22:090
255103299548,7cyclictest131rcu_preempt12:30:012
2551032995428,10cyclictest2550608-21kworker/2:108:55:022
2551032995428,10cyclictest2550608-21kworker/2:108:55:022
255102899548,17cyclictest131rcu_preempt09:30:011
2551028995418,10cyclictest23-21ksoftirqd/109:40:001
2551028995414,9cyclictest131rcu_preempt11:41:421
2551028995414,6cyclictest131rcu_preempt10:48:011
2551028995412,9cyclictest23-21ksoftirqd/107:55:001
2551024995427,12cyclictest2584817-21ssh10:11:230
2551032995345,5cyclictest2628622-21kworker/2:312:21:312
2551032995345,5cyclictest2572965-21kworker/2:010:47:422
255102899536,10cyclictest131rcu_preempt12:04:591
2551028995339,10cyclictest23-21ksoftirqd/110:24:511
2551028995339,10cyclictest23-21ksoftirqd/110:24:511
2551024995316,33cyclictest16850irq/134-800000010:40:010
2551024995312,35cyclictest12-21ksoftirqd/011:45:020
2551036995237,11cyclictest2644434-21cat12:25:023
255102899528,6cyclictest131rcu_preempt11:10:011
2551028995219,7cyclictest23-21ksoftirqd/107:40:281
2551028995214,6cyclictest131rcu_preempt12:33:391
2551028995211,6cyclictest131rcu_preempt10:29:371
2551024995223,14cyclictest2602544-21ssh10:51:080
2551032995143,5cyclictest2572965-21kworker/2:011:34:512
2551032995143,5cyclictest2544860-21kworker/2:210:19:162
2551032995143,5cyclictest2544860-21kworker/2:209:37:322
2551032995143,5cyclictest2544860-21kworker/2:209:13:112
2551032995142,5cyclictest2572965-21kworker/2:010:56:232
2551032995142,5cyclictest2544860-21kworker/2:209:48:222
2551028995115,5cyclictest131rcu_preempt11:00:011
2551028995114,6cyclictest131rcu_preempt12:20:001
2551028995113,9cyclictest23-21ksoftirqd/108:15:011
2551028995113,9cyclictest23-21ksoftirqd/108:15:011
2551028995112,6cyclictest131rcu_preempt10:20:011
2551028995112,5cyclictest131rcu_preempt09:43:441
255102899511,25cyclictest0-21swapper/110:05:021
255102899511,25cyclictest0-21swapper/110:05:011
2551028995110,15cyclictest131rcu_preempt10:10:271
255102499515,29cyclictest19350irq/34-mmc109:15:020
2551024995131,17cyclictest12-21ksoftirqd/012:36:560
2551024995127,16cyclictest12-21ksoftirqd/010:35:000
255102499512,5cyclictest131rcu_preempt09:38:410
2551024995113,17cyclictest131rcu_preempt11:01:530
2551024995113,17cyclictest131rcu_preempt11:01:520
255103299509,6cyclictest131rcu_preempt11:05:022
255103299509,6cyclictest131rcu_preempt11:05:022
2551032995043,4cyclictest2544860-21kworker/2:210:02:572
2551032995043,4cyclictest2544860-21kworker/2:210:02:562
2551032995042,5cyclictest2572965-21kworker/2:011:39:192
2551032995041,5cyclictest2572965-21kworker/2:011:41:122
255102899505,10cyclictest23-21ksoftirqd/109:51:241
2551028995019,19cyclictest23-21ksoftirqd/112:27:291
2551028995014,5cyclictest131rcu_preempt11:38:231
2551028995010,11cyclictest23-21ksoftirqd/111:15:031
2551028995010,11cyclictest23-21ksoftirqd/111:15:021
255102499502,44cyclictest2646674-21rm12:30:030
255102499502,41cyclictest2555633-21systemctl08:40:010
2551024995014,32cyclictest16850irq/134-800000011:00:010
2551036994934,12cyclictest2615253-21systemd-run11:20:003
2551036994928,16cyclictest309-21in:imuxsock11:40:003
255103699491,33cyclictest2551730-21kworker/u8:408:05:023
2551032994942,4cyclictest2572965-21kworker/2:012:05:012
2551032994941,5cyclictest2544860-21kworker/2:211:21:532
2551032994941,5cyclictest2544860-21kworker/2:211:15:072
2551032994941,5cyclictest2544860-21kworker/2:211:10:442
2551032994941,5cyclictest2544860-21kworker/2:211:10:442
2551032994941,5cyclictest2544860-21kworker/2:209:54:302
2551032994941,5cyclictest2544860-21kworker/2:209:25:542
255103299493,7cyclictest2582981-21sshd10:07:252
255102899496,6cyclictest131rcu_preempt10:52:141
2551028994925,18cyclictest23-21ksoftirqd/108:10:011
2551028994916,9cyclictest23-21ksoftirqd/107:47:241
2551028994913,5cyclictest131rcu_preempt12:10:011
2551028994911,9cyclictest131rcu_preempt11:56:001
2551028994911,9cyclictest131rcu_preempt11:55:591
2551028994911,6cyclictest131rcu_preempt07:30:031
255102499492,3cyclictest131rcu_preempt11:08:140
255102499491,4cyclictest131rcu_preempt08:20:000
2551024994912,33cyclictest16850irq/134-800000009:30:020
2551036994837,6cyclictest2561679-21sh09:20:003
2551036994829,14cyclictest3467602-21systemd-journal12:20:013
255103699481,5cyclictest131rcu_preempt11:55:013
2551036994810,6cyclictest131rcu_preempt10:15:013
255103299489,4cyclictest131rcu_preempt11:47:082
2551032994841,5cyclictest2572965-21kworker/2:010:34:042
2551032994840,5cyclictest2628622-21kworker/2:312:30:432
2551032994840,5cyclictest2572965-21kworker/2:010:35:282
2551032994840,5cyclictest2550608-21kworker/2:107:35:022
2551032994840,5cyclictest2544860-21kworker/2:208:40:022
2551032994839,6cyclictest2544860-21kworker/2:208:19:512
2551032994831,13cyclictest3467602-21systemd-journal07:55:002
2551028994814,5cyclictest131rcu_preempt07:15:001
2551028994814,5cyclictest131rcu_preempt07:15:001
2551028994812,9cyclictest131rcu_preempt10:42:531
2551028994812,6cyclictest131rcu_preempt09:24:231
255102899481,19cyclictest0-21swapper/111:47:061
255102899481,18cyclictest0-21swapper/107:35:001
2551028994811,5cyclictest131rcu_preempt11:16:051
2551028994810,6cyclictest131rcu_preempt11:02:081
2551028994810,6cyclictest131rcu_preempt11:02:071
255102499484,4cyclictest131rcu_preempt09:10:000
255102499484,4cyclictest131rcu_preempt09:10:000
2551024994831,12cyclictest275-21dbus-daemon08:05:010
2551024994821,12cyclictest2578529-21ssh09:57:210
2551036994730,11cyclictest37-21ksoftirqd/312:00:013
2551036994730,11cyclictest37-21ksoftirqd/312:00:003
2551036994717,20cyclictest37-21ksoftirqd/310:20:023
255103299479,6cyclictest131rcu_preempt10:50:412
2551032994740,5cyclictest2544860-21kworker/2:210:11:222
2551032994739,5cyclictest2628622-21kworker/2:312:18:072
2551032994739,5cyclictest2628622-21kworker/2:312:14:482
2551032994739,5cyclictest2572965-21kworker/2:010:41:532
2551032994739,5cyclictest2544860-21kworker/2:209:32:402
2551032994738,6cyclictest2544860-21kworker/2:209:43:102
2551032994738,6cyclictest2544860-21kworker/2:209:18:102
2551032994738,5cyclictest2572965-21kworker/2:010:26:072
255102899479,6cyclictest131rcu_preempt09:05:001
2551028994730,12cyclictest275-21dbus-daemon08:45:021
2551028994712,6cyclictest131rcu_preempt09:20:011
2551024994740,5cyclictest19350irq/34-mmc108:45:020
255103699468,6cyclictest131rcu_preempt12:30:013
255103699467,5cyclictest131rcu_preempt09:10:023
255103699467,5cyclictest131rcu_preempt09:10:023
255103699463,4cyclictest131rcu_preempt07:30:013
2551036994633,8cyclictest1-21systemd09:30:023
2551036994629,13cyclictest275-21dbus-daemon07:50:013
2551036994627,15cyclictest275-21dbus-daemon08:25:013
2551036994619,19cyclictest37-21ksoftirqd/309:50:363
2551036994616,24cyclictest361rcuc/312:05:023
255103299465,7cyclictest2555257-21latency_hist08:30:022
2551032994638,5cyclictest2550608-21kworker/2:109:02:542
2551032994610,5cyclictest131rcu_preempt10:23:572
2551032994610,5cyclictest131rcu_preempt10:23:562
2551028994610,5cyclictest131rcu_preempt10:06:591
2551024994629,13cyclictest2575862-21ssh09:51:140
2551024994629,12cyclictest3467602-21systemd-journal08:00:000
2551024994613,28cyclictest16850irq/134-800000011:29:160
2551024994611,31cyclictest16850irq/134-800000012:01:450
255103699454,5cyclictest131rcu_preempt10:45:003
255103699451,9cyclictest0-21swapper/310:03:043
255103699451,9cyclictest0-21swapper/310:03:033
2551036994518,19cyclictest0-21swapper/309:57:123
2551036994513,8cyclictest131rcu_preempt11:15:013
2551036994513,8cyclictest131rcu_preempt11:15:003
2551036994510,6cyclictest131rcu_preempt11:45:173
255103299456,4cyclictest131rcu_preempt12:35:582
255103299455,4cyclictest131rcu_preempt08:35:022
2551032994537,5cyclictest2572965-21kworker/2:012:08:402
2551032994537,5cyclictest2572965-21kworker/2:011:55:032
2551032994537,5cyclictest2572965-21kworker/2:011:55:022
2551032994537,5cyclictest2544860-21kworker/2:211:07:412
2551032994526,8cyclictest2544860-21kworker/2:207:15:022
2551032994526,8cyclictest2544860-21kworker/2:207:15:022
255103299451,3cyclictest131rcu_preempt08:50:012
255102899456,6cyclictest131rcu_preempt11:21:211
255102899456,18cyclictest131rcu_preempt07:20:011
255102899454,7cyclictest23-21ksoftirqd/112:25:011
2551028994536,6cyclictest23-21ksoftirqd/107:25:011
255102499454,3cyclictest131rcu_preempt08:30:020
2551024994535,7cyclictest12-21ksoftirqd/009:05:010
255103699444,5cyclictest131rcu_preempt07:45:013
2551036994427,13cyclictest3467602-21systemd-journal09:05:003
2551036994424,15cyclictest275-21dbus-daemon10:10:003
2551036994421,7cyclictest37-21ksoftirqd/309:33:353
2551032994436,5cyclictest2544860-21kworker/2:211:25:572
255102899448,6cyclictest131rcu_preempt09:00:011
255102899446,5cyclictest131rcu_preempt09:45:381
255102899443,12cyclictest23-21ksoftirqd/108:30:011
2551028994413,6cyclictest131rcu_preempt07:10:021
2551036994334,6cyclictest37-21ksoftirqd/310:55:563
2551036994329,10cyclictest1-21systemd11:25:013
2551036994316,10cyclictest37-21ksoftirqd/310:40:023
255103699431,5cyclictest131rcu_preempt12:14:593
2551036994312,10cyclictest37-21ksoftirqd/309:00:013
2551036994311,6cyclictest131rcu_preempt11:30:013
2551036994311,6cyclictest131rcu_preempt09:40:023
2551032994336,4cyclictest2544860-21kworker/2:209:00:002
2551032994330,9cyclictest275-21dbus-daemon07:25:012
2551032994328,10cyclictest275-21dbus-daemon08:45:012
255102899438,6cyclictest131rcu_preempt08:25:001
2551028994331,8cyclictest2553525-21sshd07:58:241
255102499438,3cyclictest131rcu_preempt08:55:010
255102499438,3cyclictest131rcu_preempt08:55:000
255102499436,6cyclictest2552271-21kworker/0:109:00:010
255102499431,3cyclictest131rcu_preempt07:20:010
2551036994230,8cyclictest1-21systemd10:30:003
2551036994229,10cyclictest3467602-21systemd-journal07:15:243
2551036994226,12cyclictest2622113-21wc11:35:023
2551036994217,14cyclictest2651089-21munin-run12:40:023
2551036994215,16cyclictest37-21ksoftirqd/309:13:563
2551036994214,16cyclictest37-21ksoftirqd/309:41:513
255103299426,3cyclictest131rcu_preempt07:45:022
255102899429,6cyclictest131rcu_preempt08:20:021
255102899427,7cyclictest131rcu_preempt08:05:021
2551028994229,10cyclictest2555601-21sshd08:38:241
2551028994216,17cyclictest131rcu_preempt09:09:251
2551028994216,17cyclictest131rcu_preempt09:09:241
2551024994226,11cyclictest12-21ksoftirqd/007:45:020
2551036994130,9cyclictest37-21ksoftirqd/310:51:253
2551036994127,11cyclictest37-21ksoftirqd/311:07:423
2551036994119,14cyclictest275-21dbus-daemon08:50:003
2551036994118,8cyclictest131rcu_preempt10:34:503
2551036994116,21cyclictest131rcu_preempt11:45:023
2551036994111,6cyclictest131rcu_preempt12:10:013
255103299419,6cyclictest131rcu_preempt07:46:422
2551032994124,6cyclictest131rcu_preempt07:40:002
255103299411,8cyclictest2562751-21ssh09:22:082
255103299411,8cyclictest2554880-21munin-run08:25:022
2551028994130,8cyclictest2556387-21sshd08:54:251
2551028994130,8cyclictest2556387-21sshd08:54:241
255102499411,3cyclictest131rcu_preempt07:15:010
255102499411,3cyclictest131rcu_preempt07:15:010
2551036994027,10cyclictest37-21ksoftirqd/309:48:153
2551036994017,6cyclictest131rcu_preempt08:32:243
2551024994027,10cyclictest2555438-21sed08:35:010
2551024994025,11cyclictest1-21systemd07:50:000
2551024994024,12cyclictest1-21systemd08:25:000
255102499401,3cyclictest131rcu_preempt07:25:030
2551036993914,20cyclictest131rcu_preempt10:50:003
2551036993910,20cyclictest0-21swapper/307:22:243
255103299394,4cyclictest131rcu_preempt08:10:002
255103299393,3cyclictest131rcu_preempt08:05:012
2551032993931,5cyclictest2544860-21kworker/2:207:18:462
2551028993923,9cyclictest309-21in:imuxsock08:35:011
2551024993926,10cyclictest2554306-21sshd08:13:250
2551024993926,10cyclictest2554306-21sshd08:13:240
2551024993926,10cyclictest2554191-21sed08:10:020
255102499392,3cyclictest131rcu_preempt07:30:010
2551024993921,13cyclictest1-21systemd08:45:010
255103699383,5cyclictest131rcu_preempt10:24:233
255103699383,5cyclictest131rcu_preempt10:24:223
2551036993828,7cyclictest2554294-21sshd08:10:303
2551036993828,7cyclictest2554294-21sshd08:10:303
2551036993827,9cyclictest37-21ksoftirqd/312:31:003
2551036993813,6cyclictest131rcu_preempt09:25:023
255103299386,4cyclictest131rcu_preempt07:30:012
2551028993810,7cyclictest131rcu_preempt08:50:001
255102499381,3cyclictest131rcu_preempt07:55:010
2551036993728,6cyclictest37-21ksoftirqd/307:40:033
2551036993725,9cyclictest37-21ksoftirqd/308:45:033
2551036993725,10cyclictest37-21ksoftirqd/308:55:023
2551036993725,10cyclictest37-21ksoftirqd/308:55:013
2551036993712,9cyclictest37-21ksoftirqd/308:30:013
2551024993725,8cyclictest2552302-21sed07:35:010
2551036993627,7cyclictest37-21ksoftirqd/307:15:003
2551036993627,7cyclictest37-21ksoftirqd/307:15:003
2551036993626,8cyclictest37-21ksoftirqd/307:35:013
2551032993621,11cyclictest1-21systemd08:15:012
2551032993621,11cyclictest1-21systemd08:15:012
255103699357,22cyclictest131rcu_preempt08:00:013
255103699355,5cyclictest131rcu_preempt08:40:003
2551036993521,11cyclictest37-21ksoftirqd/307:55:013
2551032993528,3cyclictest2544860-21kworker/2:207:10:022
255103699346,6cyclictest131rcu_preempt08:10:033
2551032993419,12cyclictest2553689-21sed08:00:022
2551036993220,10cyclictest37-21ksoftirqd/308:20:013
255103699301,4cyclictest131rcu_preempt07:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional