You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-22 - 18:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sun Mar 22, 2026 12:45:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
849681999286,3cyclictest16950irq/224-800000009:30:020
849681998428,31cyclictest902840-21ssh10:59:250
84968199811,45cyclictest866981-21ssh09:35:210
84968599808,56cyclictest903230-21sh11:00:021
84968199795,48cyclictest12-21ksoftirqd/010:10:540
84968199795,48cyclictest12-21ksoftirqd/010:10:540
849681997816,10cyclictest12-21ksoftirqd/010:25:020
849681997816,10cyclictest12-21ksoftirqd/010:25:020
849681997533,39cyclictest12-21ksoftirqd/012:22:070
849681997470,2cyclictest0-21swapper/011:10:020
84968199737,6cyclictest121ksoftirqd/012:08:040
849681997024,27cyclictest926085-21rm11:53:200
84968199661,9cyclictest12-21ksoftirqd/012:13:350
849681996419,39cyclictest946030-21latency_hist12:40:000
84968199635,5cyclictest121ksoftirqd/009:56:310
849681996343,4cyclictest16850irq/134-800000009:50:000
849681996225,7cyclictest861509-21rm09:22:520
849681996120,27cyclictest16850irq/134-800000010:42:360
849681996120,27cyclictest16850irq/134-800000010:42:360
849692996038,16cyclictest275-21dbus-daemon11:30:023
849689996052,5cyclictest855369-21kworker/2:312:13:272
849685996038,15cyclictest3467602-21systemd-journal08:15:001
84968599602,3cyclictest131rcu_preempt12:00:001
849681996021,11cyclictest12-21ksoftirqd/011:50:020
849692995920,10cyclictest37-21ksoftirqd/310:15:473
849692995917,12cyclictest37-21ksoftirqd/312:32:043
849685995934,13cyclictest1-21systemd07:20:011
84968199597,7cyclictest12-21ksoftirqd/011:42:520
849681995914,10cyclictest12-21ksoftirqd/010:50:400
849681995914,10cyclictest12-21ksoftirqd/010:50:400
849681995911,12cyclictest12-21ksoftirqd/010:50:000
84968199589,10cyclictest12-21ksoftirqd/007:20:000
84968199586,13cyclictest851741-21date07:50:010
849681995828,12cyclictest891560-21ssh10:32:520
84968199576,9cyclictest12-21ksoftirqd/008:50:010
849681995727,13cyclictest874540-21ssh09:53:180
849681995722,10cyclictest0-21swapper/011:26:350
849681995719,9cyclictest0-21swapper/010:02:370
84968199569,9cyclictest12-21ksoftirqd/011:33:460
84968199568,11cyclictest12-21ksoftirqd/008:15:010
84968199568,10cyclictest12-21ksoftirqd/009:45:010
84968199568,10cyclictest12-21ksoftirqd/009:45:010
84968199567,30cyclictest0-21swapper/008:55:000
84968199566,11cyclictest12-21ksoftirqd/010:10:020
84968199564,10cyclictest12-21ksoftirqd/007:15:010
849681995632,10cyclictest857791-21ssh09:14:300
84968199562,6cyclictest12-21ksoftirqd/009:16:470
849689995546,6cyclictest866642-21kworker/2:110:45:312
849689995537,13cyclictest3467602-21systemd-journal09:55:012
84968199559,7cyclictest940412-21ssh12:26:290
84968199555,13cyclictest12-21ksoftirqd/010:27:190
84968199553,43cyclictest275-21dbus-daemon09:10:010
849681995534,14cyclictest1-21systemd09:00:010
849681995534,14cyclictest1-21systemd09:00:010
84968199553,25cyclictest12-21ksoftirqd/008:40:020
849681995527,12cyclictest908662-21ssh11:12:480
849681995527,12cyclictest908662-21ssh11:12:470
849689995445,5cyclictest866642-21kworker/2:111:50:002
84968199544,10cyclictest12-21ksoftirqd/010:40:030
849681995419,8cyclictest12-21ksoftirqd/010:20:010
849681995413,9cyclictest12-21ksoftirqd/011:24:340
849681995413,7cyclictest12-21ksoftirqd/011:38:280
849681995413,7cyclictest12-21ksoftirqd/011:38:280
849689995345,5cyclictest852762-21kworker/2:209:10:392
849689995344,5cyclictest866642-21kworker/2:110:40:022
849689995335,8cyclictest855369-21kworker/2:310:26:292
849689995329,16cyclictest855369-21kworker/2:309:22:292
849681995326,12cyclictest863235-21ssh09:26:500
849681995318,8cyclictest12-21ksoftirqd/011:01:030
849689995245,4cyclictest866642-21kworker/2:112:39:412
849689995244,5cyclictest866642-21kworker/2:112:28:412
849689995244,5cyclictest866642-21kworker/2:110:00:082
849689995244,5cyclictest866642-21kworker/2:109:45:012
849689995244,5cyclictest866642-21kworker/2:109:45:012
849689995244,5cyclictest855369-21kworker/2:312:15:222
849689995244,5cyclictest855369-21kworker/2:311:24:102
849689995244,4cyclictest852762-21kworker/2:208:20:022
849689995243,6cyclictest866642-21kworker/2:111:44:332
849689995236,7cyclictest866642-21kworker/2:111:30:022
84968599521,3cyclictest131rcu_preempt09:30:011
84968199527,6cyclictest12-21ksoftirqd/012:03:580
84968199526,5cyclictest12-21ksoftirqd/008:00:010
84968199525,8cyclictest12-21ksoftirqd/011:16:110
849681995243,5cyclictest16850irq/134-800000012:30:150
849681995212,7cyclictest12-21ksoftirqd/007:55:030
849681995211,8cyclictest12-21ksoftirqd/012:19:040
849692995136,10cyclictest275-21dbus-daemon11:50:013
849692995134,11cyclictest1-21systemd09:45:023
849692995134,11cyclictest1-21systemd09:45:023
849692995114,7cyclictest131rcu_preempt07:40:013
849689995143,5cyclictest855369-21kworker/2:309:38:212
849689995143,5cyclictest855369-21kworker/2:309:20:022
849689995143,4cyclictest866642-21kworker/2:110:44:452
849689995143,4cyclictest866642-21kworker/2:110:44:442
849689995142,6cyclictest866642-21kworker/2:111:02:282
849689995142,5cyclictest866642-21kworker/2:111:35:382
849689995142,5cyclictest866642-21kworker/2:111:35:382
849689995142,5cyclictest866642-21kworker/2:110:52:092
849689995142,5cyclictest866642-21kworker/2:110:52:092
849689995128,14cyclictest866642-21kworker/2:110:32:522
849689995112,6cyclictest131rcu_preempt09:47:262
849681995112,9cyclictest12-21ksoftirqd/008:30:030
849692995036,8cyclictest854293-21grep08:40:003
849692995034,11cyclictest309-21in:imuxsock09:05:023
849692995015,5cyclictest131rcu_preempt11:11:003
849692995015,5cyclictest131rcu_preempt11:11:003
849689995042,5cyclictest866642-21kworker/2:112:23:162
849689995042,5cyclictest866642-21kworker/2:111:50:492
849689995042,5cyclictest866642-21kworker/2:110:56:042
849689995042,5cyclictest866642-21kworker/2:109:56:292
849689995042,5cyclictest855369-21kworker/2:310:24:362
849689995042,5cyclictest855369-21kworker/2:310:24:352
849689995041,5cyclictest855369-21kworker/2:309:31:392
849685995032,13cyclictest3467602-21systemd-journal07:40:001
84968199506,6cyclictest12-21ksoftirqd/009:05:010
84968199505,9cyclictest12-21ksoftirqd/011:58:060
84968199503,29cyclictest851142-21kworker/u8:008:20:030
849681995020,19cyclictest275-21dbus-daemon07:30:010
849692994930,14cyclictest275-21dbus-daemon08:55:003
84968999499,7cyclictest131rcu_preempt10:20:002
849689994941,5cyclictest866642-21kworker/2:112:31:592
849689994941,5cyclictest855369-21kworker/2:312:08:252
849689994941,5cyclictest855369-21kworker/2:311:16:332
849689994941,5cyclictest852762-21kworker/2:208:25:012
849689994940,6cyclictest855369-21kworker/2:310:07:382
849689994940,5cyclictest866642-21kworker/2:112:02:592
849685994933,10cyclictest1-21systemd08:30:011
849692994839,7cyclictest37-21ksoftirqd/311:05:583
849692994824,6cyclictest131rcu_preempt10:57:313
849692994817,8cyclictest131rcu_preempt12:15:473
849692994812,6cyclictest131rcu_preempt09:55:013
849692994810,28cyclictest37-21ksoftirqd/312:25:033
849689994840,5cyclictest866642-21kworker/2:111:58:542
849689994840,5cyclictest855369-21kworker/2:311:12:312
849689994840,5cyclictest855369-21kworker/2:311:12:312
849689994840,5cyclictest852762-21kworker/2:209:09:412
849689994839,5cyclictest866642-21kworker/2:111:30:152
849689994839,5cyclictest855369-21kworker/2:310:12:112
849689994839,5cyclictest855369-21kworker/2:310:12:102
849689994838,6cyclictest827451-21kworker/2:308:05:012
849689994836,9cyclictest3467602-21systemd-journal07:47:452
84968599486,5cyclictest131rcu_preempt10:02:211
84968199484,11cyclictest12-21ksoftirqd/008:35:010
849681994834,11cyclictest12-21ksoftirqd/007:35:000
849681994825,16cyclictest275-21dbus-daemon08:10:010
84969299477,6cyclictest131rcu_preempt12:30:013
84969299472,5cyclictest131rcu_preempt12:05:453
849689994739,5cyclictest855369-21kworker/2:309:25:072
849689994728,13cyclictest3467602-21systemd-journal07:44:592
849685994738,6cyclictest23-21ksoftirqd/111:40:531
849685994731,12cyclictest1-21systemd11:05:021
849685994730,13cyclictest1-21systemd10:55:011
849685994730,13cyclictest1-21systemd10:55:011
849685994729,13cyclictest3467602-21systemd-journal11:35:001
84968199479,27cyclictest851166-21modprobe07:40:010
84969299467,6cyclictest131rcu_preempt09:15:013
849692994622,20cyclictest37-21ksoftirqd/311:30:433
84969299461,9cyclictest0-21swapper/309:35:133
849689994638,5cyclictest855369-21kworker/2:311:09:002
849689994638,5cyclictest827451-21kworker/2:308:15:012
849689994638,5cyclictest827451-21kworker/2:308:00:012
849689994638,5cyclictest827451-21kworker/2:307:29:332
849689994637,6cyclictest852762-21kworker/2:208:50:012
849685994631,11cyclictest3467602-21systemd-journal07:14:021
84968199465,10cyclictest12-21ksoftirqd/008:25:010
84969299453,5cyclictest131rcu_preempt11:42:183
84969299452,6cyclictest131rcu_preempt11:02:263
84969299451,8cyclictest0-21swapper/310:05:393
84969299451,14cyclictest0-21swapper/311:51:073
849689994525,15cyclictest275-21dbus-daemon08:10:002
849685994534,8cyclictest909059-21sshd11:13:441
849685994534,8cyclictest909059-21sshd11:13:441
849685994529,12cyclictest275-21dbus-daemon09:55:011
849692994426,12cyclictest37-21ksoftirqd/308:20:003
84969299441,5cyclictest0-21swapper/312:03:333
849692994413,6cyclictest131rcu_preempt10:11:443
849692994413,6cyclictest131rcu_preempt10:11:433
849689994436,5cyclictest852762-21kworker/2:208:35:012
849689994426,13cyclictest30-21ksoftirqd/208:40:012
849689994419,12cyclictest30-21ksoftirqd/209:05:002
849685994429,12cyclictest896189-21sshd10:43:441
849685994429,12cyclictest896189-21sshd10:43:441
84968199445,9cyclictest12-21ksoftirqd/007:45:010
84968199444,7cyclictest12-21ksoftirqd/008:45:020
849681994410,8cyclictest0-21swapper/007:25:010
84969299438,6cyclictest131rcu_preempt09:47:013
84969299437,7cyclictest37-21ksoftirqd/312:38:203
84969299433,6cyclictest131rcu_preempt09:22:143
849692994335,5cyclictest37-21ksoftirqd/309:17:353
849692994320,4cyclictest131rcu_preempt10:25:013
849692994320,4cyclictest131rcu_preempt10:25:003
84969299431,9cyclictest0-21swapper/309:26:183
84969299431,8cyclictest0-21swapper/311:57:543
84969299431,6cyclictest0-21swapper/310:38:413
84969299431,6cyclictest0-21swapper/310:00:193
849689994334,6cyclictest827451-21kworker/2:307:30:172
84968599433,6cyclictest23-21ksoftirqd/108:40:021
849685994330,10cyclictest1-21systemd07:55:001
849685994326,12cyclictest3467602-21systemd-journal10:25:441
84968199431,12cyclictest12-21ksoftirqd/008:05:020
84968199431,11cyclictest849762-21ls07:10:020
84968199431,11cyclictest849762-21ls07:10:010
84969299423,4cyclictest131rcu_preempt11:35:453
84969299423,4cyclictest131rcu_preempt11:35:453
849692994231,8cyclictest37-21ksoftirqd/309:04:593
849692994227,11cyclictest37-21ksoftirqd/310:29:553
84969299422,5cyclictest131rcu_preempt10:49:493
849692994224,14cyclictest275-21dbus-daemon08:25:003
849692994218,20cyclictest37-21ksoftirqd/309:56:353
849692994217,11cyclictest131rcu_preempt11:20:003
84969299421,6cyclictest0-21swapper/309:30:313
84969299421,11cyclictest0-21swapper/310:44:253
84969299421,11cyclictest0-21swapper/310:44:243
849689994234,5cyclictest827451-21kworker/2:307:18:442
849689994227,11cyclictest3467602-21systemd-journal07:51:442
84968599424,4cyclictest131rcu_preempt12:10:501
84968599423,3cyclictest131rcu_preempt10:15:021
84968599423,3cyclictest131rcu_preempt10:15:021
849685994229,11cyclictest23-21ksoftirqd/112:01:011
849685994228,11cyclictest1-21systemd12:40:011
849685994223,13cyclictest275-21dbus-daemon09:35:001
84969299418,7cyclictest131rcu_preempt12:15:033
84969299414,10cyclictest37-21ksoftirqd/307:35:003
84969299411,7cyclictest0-21swapper/310:34:333
849692994110,7cyclictest131rcu_preempt08:50:003
849689994110,4cyclictest131rcu_preempt08:57:162
849689994110,4cyclictest131rcu_preempt08:57:152
84968599418,5cyclictest131rcu_preempt12:21:561
84968599414,6cyclictest131rcu_preempt11:20:001
849685994131,7cyclictest23-21ksoftirqd/110:30:391
849685994131,7cyclictest23-21ksoftirqd/109:10:001
849685994128,10cyclictest1-21systemd10:50:001
849685994127,10cyclictest924622-21systemd-run11:50:001
849685994125,11cyclictest23-21ksoftirqd/108:20:441
84968599412,3cyclictest131rcu_preempt09:58:331
84968599411,4cyclictest131rcu_preempt10:40:001
84969299409,5cyclictest131rcu_preempt10:50:233
84969299409,5cyclictest131rcu_preempt10:50:223
84969299408,4cyclictest131rcu_preempt08:30:003
84969299403,6cyclictest131rcu_preempt11:20:303
84969299402,6cyclictest131rcu_preempt09:00:013
84969299402,6cyclictest131rcu_preempt09:00:013
849692994023,12cyclictest275-21dbus-daemon08:35:003
84969299402,11cyclictest37-21ksoftirqd/307:56:443
849692994016,6cyclictest131rcu_preempt07:25:003
84968599404,4cyclictest131rcu_preempt12:10:011
84968599404,3cyclictest131rcu_preempt09:23:451
84968599404,3cyclictest131rcu_preempt09:10:431
84968599403,4cyclictest131rcu_preempt09:00:021
84968599403,4cyclictest131rcu_preempt09:00:011
84968599402,6cyclictest23-21ksoftirqd/108:10:021
849685994026,11cyclictest23-21ksoftirqd/109:43:451
849685994026,11cyclictest23-21ksoftirqd/109:43:451
84968599401,6cyclictest131rcu_preempt10:19:471
84968599401,3cyclictest131rcu_preempt12:18:541
849692993928,8cyclictest275-21dbus-daemon08:45:013
849692993923,12cyclictest309-21in:imuxsock07:10:023
849692993917,18cyclictest37-21ksoftirqd/308:05:013
84968999392,9cyclictest30-21ksoftirqd/207:35:262
84968599398,6cyclictest131rcu_preempt09:16:581
84968599393,5cyclictest131rcu_preempt12:34:091
84968599392,6cyclictest131rcu_preempt07:35:011
84968599391,3cyclictest131rcu_preempt12:30:021
849685993912,6cyclictest131rcu_preempt11:05:591
849692993828,6cyclictest851688-21cron07:49:593
84968999389,5cyclictest131rcu_preempt08:28:392
849689993822,5cyclictest30-21ksoftirqd/208:55:012
849689993821,4cyclictest30-21ksoftirqd/207:25:012
849689993821,13cyclictest275-21dbus-daemon07:15:002
84968599383,6cyclictest131rcu_preempt11:25:011
84968599383,3cyclictest131rcu_preempt10:05:091
84968599383,3cyclictest131rcu_preempt09:40:021
849685993826,8cyclictest926826-21cat11:55:011
84968599382,4cyclictest131rcu_preempt11:35:391
84968599382,4cyclictest131rcu_preempt11:35:381
849685993824,11cyclictest887310-21ssh10:23:011
849685993824,11cyclictest887310-21ssh10:23:011
84968599382,3cyclictest131rcu_preempt08:19:591
84968599374,4cyclictest131rcu_preempt08:05:001
84968599373,3cyclictest131rcu_preempt09:50:021
849685993724,9cyclictest275-21dbus-daemon07:45:011
84968599372,3cyclictest131rcu_preempt08:45:021
84968599371,3cyclictest131rcu_preempt11:27:201
84968599371,3cyclictest131rcu_preempt07:30:011
84969299365,9cyclictest37-21ksoftirqd/307:10:013
84969299365,9cyclictest37-21ksoftirqd/307:10:003
84969299362,5cyclictest131rcu_preempt07:45:023
84968999365,4cyclictest131rcu_preempt08:45:012
84968599364,3cyclictest131rcu_preempt09:00:031
849685993625,7cyclictest23-21ksoftirqd/108:50:011
849685993621,11cyclictest850465-21cut07:25:011
849692993517,13cyclictest37-21ksoftirqd/307:55:023
84968599352,3cyclictest131rcu_preempt08:32:451
849692993417,13cyclictest3467602-21systemd-journal07:20:003
84968599344,3cyclictest131rcu_preempt07:49:451
849685993424,7cyclictest852185-21sshd07:59:281
84968599342,3cyclictest131rcu_preempt08:55:021
84969299339,5cyclictest131rcu_preempt08:15:023
849692993322,7cyclictest37-21ksoftirqd/307:30:023
84969299317,4cyclictest131rcu_preempt08:07:413
849685993121,8cyclictest23-21ksoftirqd/107:10:011
849685993121,8cyclictest23-21ksoftirqd/107:10:001
84968999293,3cyclictest131rcu_preempt07:10:012
84968999293,3cyclictest131rcu_preempt07:10:002
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional